btrfs: use single bulk copy operations when logging directories

When logging a directory and inserting a batch of directory items, we are
copying the data of each item from a leaf in the fs/subvolume tree to a
leaf in a log tree, separately. This is not really needed, since we are
copying from a contiguous memory area into another one, so we can use a
single copy operation to copy all items at once.

This patch is part of a small patchset that is comprised of the following
patches:

  btrfs: loop only once over data sizes array when inserting an item batch
  btrfs: unexport setup_items_for_insert()
  btrfs: use single bulk copy operations when logging directories

This is patch 3/3.

The following test was used to compare performance of a branch without the
patchset versus one branch that has the whole patchset applied:

  $ cat dir-fsync-test.sh
  #!/bin/bash

  DEV=/dev/nvme0n1
  MNT=/mnt/nvme0n1

  NUM_NEW_FILES=1000000
  NUM_FILE_DELETES=1000
  LEAF_SIZE=16K

  mkfs.btrfs -f -n $LEAF_SIZE $DEV
  mount -o ssd $DEV $MNT

  mkdir $MNT/testdir

  for ((i = 1; i <= $NUM_NEW_FILES; i++)); do
      echo -n > $MNT/testdir/file_$i
  done

  # Fsync the directory, this will log the new dir items and the inodes
  # they point to, because these are new inodes.
  start=$(date +%s%N)
  xfs_io -c "fsync" $MNT/testdir
  end=$(date +%s%N)

  dur=$(( (end - start) / 1000000 ))
  echo "dir fsync took $dur ms after adding $NUM_NEW_FILES files"

  # sync to force transaction commit and wipeout the log.
  sync

  del_inc=$(( $NUM_NEW_FILES / $NUM_FILE_DELETES ))
  for ((i = 1; i <= $NUM_NEW_FILES; i += $del_inc)); do
      rm -f $MNT/testdir/file_$i
  done

  # Fsync the directory, this will only log dir items, there are no
  # dentries pointing to new inodes.
  start=$(date +%s%N)
  xfs_io -c "fsync" $MNT/testdir
  end=$(date +%s%N)

  dur=$(( (end - start) / 1000000 ))
  echo "dir fsync took $dur ms after deleting $NUM_FILE_DELETES files"

  umount $MNT

The tests were run on a non-debug kernel (Debian's default kernel config)
and were the following:

*** with a leaf size of 16K, before patchset ***

dir fsync took 8482 ms after adding 1000000 files
dir fsync took 166 ms after deleting 1000 files

*** with a leaf size of 16K, after patchset ***

dir fsync took 8196 ms after adding 1000000 files  (-3.4%)
dir fsync took 143 ms after deleting 1000 files    (-14.9%)

*** with a leaf size of 64K, before patchset ***

dir fsync took 12851 ms after adding 1000000 files
dir fsync took 466 ms after deleting 1000 files

*** with a leaf size of 64K, after  patchset ***

dir fsync took 12287 ms after adding 1000000 files (-4.5%)
dir fsync took 414 ms after deleting 1000 files    (-11.8%)

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>

diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index f3688e753c36a30e9a45e8cf79f0679b41a296fa..641e2b5a3b6494f3a98171c03fab062f7a9996c6 100644 (file)
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -3670,6 +3670,8 @@ static int flush_dir_items_batch(struct btrfs_trans_handle *trans,
        char *ins_data = NULL;
        struct btrfs_item_batch batch;
        struct extent_buffer *dst;
+       unsigned long src_offset;
+       unsigned long dst_offset;
        struct btrfs_key key;
        u32 item_size;
        int ret;
@@ -3713,16 +3715,19 @@ static int flush_dir_items_batch(struct btrfs_trans_handle *trans,
                goto out;
 
        dst = dst_path->nodes[0];
-       for (i = 0; i < count; i++) {
-               unsigned long src_offset;
-               unsigned long dst_offset;
-
-               dst_offset = btrfs_item_ptr_offset(dst, dst_path->slots[0]);
-               src_offset = btrfs_item_ptr_offset(src, start_slot + i);
-               copy_extent_buffer(dst, src, dst_offset, src_offset,
-                                  batch.data_sizes[i]);
-               dst_path->slots[0]++;
-       }
+       /*
+        * Copy all the items in bulk, in a single copy operation. Item data is
+        * organized such that it's placed at the end of a leaf and from right
+        * to left. For example, the data for the second item ends at an offset
+        * that matches the offset where the data for the first item starts, the
+        * data for the third item ends at an offset that matches the offset
+        * where the data of the second items starts, and so on.
+        * Therefore our source and destination start offsets for copy match the
+        * offsets of the last items (highest slots).
+        */
+       dst_offset = btrfs_item_ptr_offset(dst, dst_path->slots[0] + count - 1);
+       src_offset = btrfs_item_ptr_offset(src, start_slot + count - 1);
+       copy_extent_buffer(dst, src, dst_offset, src_offset, batch.total_data_size);
        btrfs_release_path(dst_path);
 out:
        kfree(ins_data);