Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
0a16f44
bcachefs: btree_node_(rewrite|update_key) cleanup
Feb 25, 2025
1c6b1eb
bcachefs: Repair btree node write errors immediately
Feb 25, 2025
660eec6
bcachefs: bs > ps support
Feb 20, 2025
08ff1b2
bcachefs: check_bp_exists() check for backpointers for stale pointers
Feb 11, 2025
998b19d
bcachefs: Fix missing increment of move_extent_write counter
Jan 29, 2025
31fa367
bcachefs: Don't inc io_(read|write) counters for moves
Jan 30, 2025
7f7ecbb
bcachefs: Move write_points to debugfs
Jan 22, 2025
ffe4421
bcachefs: Separate running/runnable in wp stats
Jan 22, 2025
2f5291d
bcachefs: enum bch_persistent_counters_stable
Jan 30, 2025
f063fad
bcachefs: BCH_COUNTER_bucket_discard_fast
Jan 30, 2025
9a3f6c8
bcachefs: BCH_IOCTL_QUERY_COUNTERS
Jan 27, 2025
c0a5117
bcachefs: bch2_data_update_inflight_to_text()
Dec 31, 2024
7b11330
bcachefs: kill bch_read_bio.devs_have
Jan 17, 2025
9d9adc2
bcachefs: Avoid holding btree locks when blocking on IO
Jan 16, 2025
3d6c203
bcachefs: x-macroize BCH_READ flags
Jan 17, 2025
9b8e1ef
bcachefs: Rename BCH_WRITE flags fer consistency with other x-macros …
Jan 19, 2025
80da5c2
bcachefs: rbio_init_fragment()
Jan 14, 2025
8839275
bcachefs: rbio_init() cleanup
Jan 15, 2025
71f7943
bcachefs: data_update now embeds bch_read_bio
Jan 15, 2025
b1e4725
bcachefs: promote_op uses embedded bch_read_bio
Jan 16, 2025
b4bc383
bcachefs: bch2_update_unwritten_extent() no longer depends on wbio
Jan 17, 2025
266e3c3
bcachefs: cleanup redundant code around data_update_op initialization
Jan 16, 2025
c320121
bcachefs: Be stricter in bch2_read_retry_nodecode()
Jan 16, 2025
caee3f6
bcachefs: Promotes should use BCH_WRITE_only_specified_devs
Jan 19, 2025
0dc4a96
bcachefs: Self healing writes are BCH_WRITE_alloc_nowait
Jan 18, 2025
ae1a494
bcachefs: Rework init order in bch2_data_update_init()
Jan 19, 2025
f0e1a90
bcachefs: Bail out early on alloc_nowait data updates
Jan 19, 2025
86ac1d4
bcachefs: Don't start promotes from bch2_rbio_free()
Jan 18, 2025
d46eb52
bcachefs: Don't self-heal if a data update is already rewriting
Jan 20, 2025
8985dd0
bcachefs: Internal reads can now correct errors
Dec 30, 2024
13fe07e
bcachefs: backpointer_get_key() doesn't pull in btree node
Dec 28, 2024
d61a5ea
bcachefs: bch2_btree_node_rewrite_pos()
Dec 29, 2024
0848e7b
bcachefs: bch2_move_data_phys()
Dec 28, 2024
0deb8dc
bcachefs: __bch2_move_data_phys() now uses bch2_btree_node_rewrite_pos()
Dec 29, 2024
17ce9b1
bcachefs: bch2_bkey_pick_read_device() can now specify a device
Dec 29, 2024
b3f946d
bcachefs: bch2_btree_node_scrub()
Dec 30, 2024
eb772c7
bcachefs: Scrub
Dec 29, 2024
aeb607b
bcachefs: Fix subtraction underflow
alanskind Jan 27, 2025
e25420b
bcachefs: Read/move path counter work
Feb 4, 2025
73f1462
bcachefs: Convert migrate to move_data_phys()
Jan 23, 2025
96565df
bcachefs: bch2_indirect_extent_missing_error() prints path, not just …
Feb 7, 2025
f98fa63
bcachefs: bch2_inum_offset_err_msg_trans() no longer handles transact…
Feb 7, 2025
eb54c7b
bcachefs: Factor out progress.[ch]
Feb 6, 2025
6e058f4
bcachefs: Add a progress indicator to bch2_dev_data_drop()
Feb 6, 2025
122ab2d
bcachefs: add progress indicator to check_allocations
Feb 7, 2025
5015709
bcachefs: Kill journal_res_state.unwritten_idx
Jan 23, 2025
b24ca6d
bcachefs: Kill journal_res.idx
Jan 23, 2025
115b63b
bcachefs: Don't touch journal_buf->data->seq in journal_res_get
Jan 23, 2025
111011d
bcachefs: Free journal bufs when not in use
Feb 6, 2025
cad15a9
bcachefs: Increase JOURNAL_BUF_NR
Jan 23, 2025
d93af77
bcachefs: Ignore backpointers to stripes in ec_stripe_update_extents()
Feb 8, 2025
7253724
bcachefs: Add comment explaining why asserts in invalidate_one_bucket…
Feb 7, 2025
fc030e0
bcachefs: Add time_stat for btree writes
Jan 27, 2025
cf3a136
bcachefs: bch2_bkey_ptr_data_type() now correctly returns cached for …
Feb 7, 2025
b2455d3
bcachefs: bch2_blacklist_entries_gc cleanup
Jan 28, 2025
bff7393
bcachefs: EYTZINGER_DEBUG fix
Jan 28, 2025
d0c66f9
bcachefs: eytzinger self tests: loop cleanups
Nov 26, 2024
843b23b
bcachefs: eytzinger self tests: missing newline termination
Nov 26, 2024
118dd81
bcachefs: eytzinger self tests: fix cmp_u16 typo
Nov 26, 2024
fb982fc
bcachefs: eytzinger[01]_test improvement
Jan 26, 2025
2283ad9
bcachefs: eytzinger0_find_test improvement
Jan 26, 2025
50ce006
bcachefs: add eytzinger0_for_each_prev
Jan 27, 2025
2c73835
bcachefs: improve eytzinger0_find_le self test
Jan 26, 2025
16d20bc
bcachefs: convert eytzinger0_find_le to be 1-based
Jan 28, 2025
e3b06c0
bcachefs: simplify eytzinger0_find_le
Jan 27, 2025
40e8034
bcachefs: add eytzinger0_find_gt self test
Jan 27, 2025
fce0efb
bcachefs: implement eytzinger0_find_gt directly
Jan 27, 2025
28d3c2b
bcachefs: implement eytzinger0_find_ge directly
Jan 27, 2025
ff7c2d1
bcachefs: add eytzinger0_find_ge self test
Jan 27, 2025
a293e3b
bcachefs: Add eytzinger0_find self test
Feb 1, 2025
489345f
bcachefs: convert eytzinger0_find to be 1-based
Jan 28, 2025
f79ddc3
bcachefs: convert eytzinger sort to be 1-based (1)
Nov 27, 2024
be910d5
bcachefs: convert eytzinger sort to be 1-based (2)
Jan 27, 2025
5f112da
bcachefs: eytzinger1_{next,prev} cleanup
Jan 28, 2025
6bf9a20
bcachefs: metadata_target is not an inode option
Feb 10, 2025
0f2c463
bcachefs: bch2_write_op_error() now prints info about data update
Feb 10, 2025
f6e2e0d
bcachefs: minor journal errcode cleanup
Feb 12, 2025
0d7bb23
bcachefs: bch2_lru_change() checks for no-op
Feb 10, 2025
e9ffa83
bcachefs: s/BCH_LRU_FRAGMENTATION_START/BCH_LRU_BUCKET_FRAGMENTATION/
Feb 10, 2025
45e38cd
bcachefs: decouple bch2_lru_check_set() from alloc btree
Feb 10, 2025
ce44ddd
bcachefs: Rework bch2_check_lru_key()
Feb 10, 2025
04eb250
bcachefs: bch2_trigger_stripe_ptr() no longer uses ec_stripes_heap_lock
Feb 11, 2025
3afccf5
bcachefs: Better trigger ordering
Feb 11, 2025
5e87769
bcachefs: rework bch2_trans_commit_run_triggers()
Feb 11, 2025
1d57764
bcachefs: bcachefs_metadata_version_cached_backpointers
Feb 7, 2025
6bc36e8
bcachefs: Invalidate cached data by backpointers
Feb 7, 2025
182295b
bcachefs: Advance bch_alloc.oldest_gen if no stale pointers
Feb 8, 2025
7d82b82
bcachefs: bcachefs_metadata_version_stripe_backpointers
Feb 7, 2025
ffff334
bcachefs: bcachefs_metadata_version_stripe_lru
Feb 8, 2025
2f1c23c
bcachefs: Kill dirent_occupied_size() in rename path
Feb 20, 2025
fb61d18
bcachefs: Kill dirent_occupied_size() in create path
Feb 20, 2025
dceab08
bcachefs: Split out dirent alloc and name initialization
misyltoad Aug 13, 2023
fd25a09
bcachefs: bcachefs_metadata_version_casefolding
misyltoad Aug 13, 2023
d130327
bcachefs: sysfs internal/trigger_btree_updates
Feb 13, 2025
83146c7
Documentation: bcachefs: casefolding: Do not italicize NUL
bagasme Feb 22, 2025
3639f88
Documentation: bcachefs: casefolding: Fix dentry/dcache consideration…
bagasme Feb 22, 2025
7cbc0e1
Documentation: bcachefs: casefolding: Use bullet list for dirent stru…
bagasme Feb 22, 2025
0db2c6f
Documentation: bcachefs: Add casefolding toctree entry
bagasme Feb 22, 2025
c401d6e
Documentation: bcachefs: Split index toctree
bagasme Feb 24, 2025
9cefccb
Documentation: bcachefs: SubmittingPatches: Demote section headings
bagasme Feb 24, 2025
38640a8
Documentation: bcachefs: SubmittingPatches: Convert footnotes to reST…
bagasme Feb 24, 2025
2ba82cb
bcachefs: bch2_request_incompat_feature() now returns error code
Feb 28, 2025
cd3df29
bcachefs: bcachefs_metadata_version_extent_flags
Feb 25, 2025
4466d8e
bcachefs: give bch2_write_super() a proper error code
Mar 1, 2025
7e5824f
bcachefs: data_update now checks for extents that can't be moved
Feb 28, 2025
728bd6d
bcachefs: Fix read path io_ref handling
Feb 28, 2025
dbde371
bcachefs: bch2_account_io_completion()
Feb 28, 2025
5847b31
bcachefs: Finish bch2_account_io_completion() conversions
Feb 28, 2025
103a395
Fix typos
Gelma Dec 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 25 additions & 18 deletions Documentation/filesystems/bcachefs/SubmittingPatches.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
Submitting patches to bcachefs:
===============================
Submitting patches to bcachefs
==============================

Here are suggestions for submitting patches to bcachefs subsystem.

Submission checklist
--------------------

Patches must be tested before being submitted, either with the xfstests suite
[0], or the full bcachefs test suite in ktest [1], depending on what's being
[0]_, or the full bcachefs test suite in ktest [1]_, depending on what's being
touched. Note that ktest wraps xfstests and will be an easier method to running
it for most users; it includes single-command wrappers for all the mainstream
in-kernel local filesystems.
Expand All @@ -26,21 +31,21 @@ considered out of date), but try not to deviate too much without reason.
Focus on writing code that reads well and is organized well; code should be
aesthetically pleasing.

CI:
===
CI
--

Instead of running your tests locally, when running the full test suite it's
prefereable to let a server farm do it in parallel, and then have the results
in a nice test dashboard (which can tell you which failures are new, and
presents results in a git log view, avoiding the need for most bisecting).

That exists [2], and community members may request an account. If you work for
That exists [2]_, and community members may request an account. If you work for
a big tech company, you'll need to help out with server costs to get access -
but the CI is not restricted to running bcachefs tests: it runs any ktest test
(which generally makes it easy to wrap other tests that can run in qemu).

Other things to think about:
============================
Other things to think about
---------------------------

- How will we debug this code? Is there sufficient introspection to diagnose
when something starts acting wonky on a user machine?
Expand Down Expand Up @@ -79,20 +84,22 @@ Other things to think about:
tested? (Automated tests exists but aren't in the CI, due to the hassle of
disk image management; coordinate to have them run.)

Mailing list, IRC:
==================
Mailing list, IRC
-----------------

Patches should hit the list [3], but much discussion and code review happens on
IRC as well [4]; many people appreciate the more conversational approach and
quicker feedback.
Patches should hit the list [3]_, but much discussion and code review happens
on IRC as well [4]_; many people appreciate the more conversational approach
and quicker feedback.

Additionally, we have a lively user community doing excellent QA work, which
exists primarily on IRC. Please make use of that resource; user feedback is
important for any nontrivial feature, and documenting it in commit messages
would be a good idea.

[0]: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
[1]: https://evilpiepirate.org/git/ktest.git/
[2]: https://evilpiepirate.org/~testdashboard/ci/
[3]: [email protected]
[4]: irc.oftc.net#bcache, #bcachefs-dev
.. rubric:: References

.. [0] git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
.. [1] https://evilpiepirate.org/git/ktest.git/
.. [2] https://evilpiepirate.org/~testdashboard/ci/
.. [3] [email protected]
.. [4] irc.oftc.net#bcache, #bcachefs-dev
90 changes: 90 additions & 0 deletions Documentation/filesystems/bcachefs/casefolding.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
.. SPDX-License-Identifier: GPL-2.0

Casefolding
===========

bcachefs has support for case-insensitive file and directory
lookups using the regular `chattr +F` (`S_CASEFOLD`, `FS_CASEFOLD_FL`)
casefolding attributes.

The main usecase for casefolding is compatibility with software written
against other filesystems that rely on casefolded lookups
(eg. NTFS and Wine/Proton).
Taking advantage of file-system level casefolding can lead to great
loading time gains in many applications and games.

Casefolding support requires a kernel with the `CONFIG_UNICODE` enabled.
Once a directory has been flagged for casefolding, a feature bit
is enabled on the superblock which marks the filesystem as using
casefolding.
When the feature bit for casefolding is enabled, it is no longer possible
to mount that filesystem on kernels without `CONFIG_UNICODE` enabled.

On the lookup/query side: casefolding is implemented by allocating a new
string of `BCH_NAME_MAX` length using the `utf8_casefold` function to
casefold the query string.

On the dirent side: casefolding is implemented by ensuring the `bkey`'s
hash is made from the casefolded string and storing the cached casefolded
name with the regular name in the dirent.

The structure looks like this:

* Regular: [dirent data][regular name][nul][nul]...
* Casefolded: [dirent data][reg len][cf len][regular name][casefolded name][nul][nul]...

(Do note, the number of NULs here is merely for illustration; their count can
vary per-key, and they may not even be present if the key is aligned to
`sizeof(u64)`.)

This is efficient as it means that for all file lookups that require casefolding,
it has identical performance to a regular lookup:
a hash comparison and a `memcmp` of the name.

Rationale
---------

Several designs were considered for this system:
One was to introduce a dirent_v2, however that would be painful especially as
the hash system only has support for a single key type. This would also need
`BCH_NAME_MAX` to change between versions, and a new feature bit.

Another option was to store without the two lengths, and just take the length of
the regular name and casefolded name contiguously / 2 as the length. This would
assume that the regular length == casefolded length, but that could potentially
not be true, if the uppercase unicode glyph had a different UTF-8 encoding than
the lowercase unicode glyph.
It would be possible to disregard the casefold cache for those cases, but it was
decided to simply encode the two string lengths in the key to avoid random
performance issues if this edgecase was ever hit.

The option settled on was to use a free-bit in d_type to mark a dirent as having
a casefold cache, and then treat the first 4 bytes the name block as lengths.
You can see this in the `d_cf_name_block` member of union in `bch_dirent`.

The feature bit was used to allow casefolding support to be enabled for the majority
of users, but some allow users who have no need for the feature to still use bcachefs as
`CONFIG_UNICODE` can increase the kernel side a significant amount due to the tables used,
which may be decider between using bcachefs for eg. embedded platforms.

Other filesystems like ext4 and f2fs have a super-block level option for casefolding
encoding, but bcachefs currently does not provide this. ext4 and f2fs do not expose
any encodings than a single UTF-8 version. When future encodings are desirable,
they will be added trivially using the opts mechanism.

dentry/dcache considerations
----------------------------

Currently, in casefolded directories, bcachefs (like other filesystems) will not cache
negative dentry's.

This is because currently doing so presents a problem in the following scenario:

- Lookup file "blAH" in a casefolded directory
- Creation of file "BLAH" in a casefolded directory
- Lookup file "blAH" in a casefolded directory

This would fail if negative dentry's were cached.

This is slightly suboptimal, but could be fixed in future with some vfs work.

20 changes: 19 additions & 1 deletion Documentation/filesystems/bcachefs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,28 @@
bcachefs Documentation
======================

Subsystem-specific development process notes
--------------------------------------------

Development notes specific to bcachefs. These are intended to supplement
:doc:`general kernel development handbook </process/index>`.

.. toctree::
:maxdepth: 2
:maxdepth: 1
:numbered:

CodingStyle
SubmittingPatches

Filesystem implementation
-------------------------

Documentation for filesystem features and their implementation details.
At this moment, only a few of these are described here.

.. toctree::
:maxdepth: 1
:numbered:

casefolding
errorcodes
2 changes: 1 addition & 1 deletion fs/bcachefs/Kconfig
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ config BCACHEFS_ERASURE_CODING
depends on BCACHEFS_FS
select QUOTACTL
help
This enables the "erasure_code" filesysystem and inode option, which
This enables the "erasure_code" filesystem and inode option, which
organizes data into reed-solomon stripes instead of ordinary
replication.

Expand Down
1 change: 1 addition & 0 deletions fs/bcachefs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ bcachefs-y := \
nocow_locking.o \
opts.o \
printbuf.o \
progress.o \
quota.o \
rebalance.o \
rcu_pending.o \
Expand Down
5 changes: 3 additions & 2 deletions fs/bcachefs/acl.c
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ static struct posix_acl *bch2_acl_from_disk(struct btree_trans *trans,
return NULL;

acl = allocate_dropping_locks(trans, ret,
posix_acl_alloc(count, _gfp));
posix_acl_alloc(count, GFP_KERNEL));
if (!acl)
return ERR_PTR(-ENOMEM);
if (ret) {
Expand Down Expand Up @@ -422,7 +422,8 @@ int bch2_acl_chmod(struct btree_trans *trans, subvol_inum inum,
if (ret)
goto err;

ret = allocate_dropping_locks_errcode(trans, __posix_acl_chmod(&acl, _gfp, mode));
ret = allocate_dropping_locks_errcode(trans,
__posix_acl_chmod(&acl, GFP_KERNEL, mode));
if (ret)
goto err;

Expand Down
Loading