Skip to content

Introduce two new docs covering codecs and converters#7211

Open
parlough wants to merge 12 commits into
mainfrom
feat/converters-and-codecs
Open

Introduce two new docs covering codecs and converters#7211
parlough wants to merge 12 commits into
mainfrom
feat/converters-and-codecs

Conversation

@parlough
Copy link
Copy Markdown
Member

@parlough parlough commented Mar 10, 2026

Inspired by the pre-existing Converters and codecs page, introduce two new articles that are more focused than the original and follow our current standards for writing and structure.

  • Use converters and codecs: Introduces Dart developers to codecs and converters, how to use them on their own, how to compose them, and how to use them with streams.
  • Build custom converters and codecs: A guide on how to build your own converters and codecs and some considerations to make when doing so.

Resolves #5175

@dart-github-bot
Copy link
Copy Markdown
Collaborator

dart-github-bot commented Mar 10, 2026

Visit the preview URL for this PR (updated for commit d833eff):

https://dart-dev--pr7211-feat-converters-and-codecs-6xdboazt.web.app

@parlough parlough requested review from conooi and lrhn March 16, 2026 15:58
@parlough parlough marked this pull request as ready for review March 16, 2026 15:59
gemini-code-assist[bot]

This comment was marked as outdated.

@parlough parlough changed the title Split and rewrite coverage on converters and codecs Introduce two new docs covering codecs and converters Mar 16, 2026
@parlough
Copy link
Copy Markdown
Member Author

@lrhn would you be able to take a look at these two pages covering converters and codecs? I'd appreciate any insights you have!

@conooi When you have a chance, this PR is ready for an initial look.

@lamek This PR includes a variety of code excerpts if you'd like to take a look.

Thanks all! I appreciate it :)

Copy link
Copy Markdown
Contributor

@conooi conooi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! Thanks, Parker!!

Copy link
Copy Markdown
Member

@lrhn lrhn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Super update.

I tend to want to over-communicate all the details, so take my suggestions with an unhealthy amount of salt.

/// Shifts the specified [codeUnit] by
/// [shift] positions in the alphabet.
///
/// Only shifts lowercase ASCII letters (a-z).
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Can farily easily do upper case too, just create

  const a = 0x61;
  const z = 0x7a;
  const caseDelta = 0x20;
  final lowerCodeUnit = codeUnit | caseDelta;
  if (lowerCodeUnit >= a && lowerCodeUnit <= z) {
    final letterNumber = lowerCodeUnit - a;
    return codeUnit - letterNumber + (letterNumber + shift).remainder(26);
  }

Easier than having to explain why it doesn't do it.)

void addSlice(String chunk, int start, int end, bool isLast) {
final buffer = StringBuffer();
for (var i = start; i < end; i++) {
buffer.writeCharCode(_shiftCodeUnit(chunk.codeUnitAt(i), _shift));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I'd have a String _shiftCodeUnits(String input, int shift) {...} helper. You write this code repeatedly. Well, at least twice.)

const a = 0x61;
const z = 0x7A;
if (codeUnit >= a && codeUnit <= z) {
return a + (codeUnit - a + shift) % 26;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use .remainder(26) instead of % 26.

Using .remainder(26) guarantees a result in the range 0..25.

There is no guard on the shift value, so it can potentially be negative, which can make (codeUnit - a + shift) negative and then %26 gives a negative result.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm coming back to this a few months later (thanks for the thorough review), and I wanted to confirm this suggestion.

https://api.dart.dev/dart-core/num/operator_modulo.html suggests for the % operator, "The sign of the returned value r is always positive". Doesn't that match what we want here? A bit of a contrast compared to many other languages like Java and JS though, so perhaps it's worth me explicitly calling out the behavior of %?

@@ -0,0 +1,172 @@
import 'dart:convert';

// #docregion caesar-encoder-sink
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(For readability of the file itself, I'd put the CaesarCodec first, then CaesarConverter, and then the private classes they depend on. But that's just, like, my way of structuring things: Most important first, things it depends on below. If you don't need to know the details of the dependencies, you can just not scroll down, and if you need to find a detail, you only need to look below, not above.

If this file only exists as a file to extract chunks from for the article, then its readability probably isn't important.)


// #docregion caesar-encoder-sink
/// A [StringConversionSink] that applies a Caesar cipher [_shift] and
/// forwards the result to the [_output] sink.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Single-line first paragraph.
Consider:

/// A [StringConversionSink] that applies a Casar cipher.
///
/// Shifts inputs by [_shift] and emits the shifted strings on
/// [_output].

// Read a file as a stream of bytes, decode UTF-8 into a string,
// then split the single string into lines.
final lines = File('data.txt')
.openRead() //
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why // here? Trying to coerce the formatter?

// #docregion stream-lines
void streamLinesExample() async {
// Read a file as a stream of bytes, decode UTF-8 into a string,
// then split the single string into lines.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really a single string. That would be the description of:

    final bytes = File('data.txt').readAsBytesSync();
    final string = utf8.decode(bytes);
    final lines = const LineSplitter().convert(string);

The streaming version specifically doesn't create a single string.


// #docregion encoding-lookup
void encodingLookupExample() {
final encoding = Encoding.getByName('utf-8');
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd put a ! after the lookup instead of a ?. in the use below.

`Encoding` also provides a default implementation of [`decodeStream`][].
In contrast to `Stream.transform` which returns a `Stream<String>`,
the `decodeStream` method reads a byte stream and
returns a single decoded `String` (wrapped in a `Future`).
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

returns ... (wrapped in a Future) -> asynchronously returns ...

Or just -> return ...

People know that "returns Future<String>" means "returns String, but later".

If the conversion is about how data is _represented_,
such as with bytes, strings, or compressed data,
the codec pattern is a natural fit.
If the conversion is about how data is _structured_,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Well, that is a very good description of what I tried to say above. Say the same thing there!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update or remove 'Converters and codecs' article

4 participants