TTL/Expiration setting & propagation #75

dfinkel · 2025-05-30T21:46:56Z

Add support for expiration-tracking within the lru implementation, and hook that up through the Galaxy type.

Add a new GalaxyWithBackendInfo method to Universe to allow users to provide hydration callbacks that return BackendGetInfo that includes an optional expiration. (the zero-valued time.Time is considered "no expiration").

The lru support leverages a heap to track the next expiration. It evicts all expired entries before evicting anything.

go 1.25 is 3 months old and the 1.26 freeze is about to begin. Bump go.mod to 1.24 so we can use weak pointers without having to add new build tags.

Leverage the container/heap stdlib package to manage a slice-backed heap for a first-pass.

sergiosalvatore

Few small things...

sergiosalvatore · 2025-11-18T15:39:05Z

galaxycache.go

+	return BackendGetInfo{}, l.be.Get(ctx, key, dest)
+}
+
+// BackendGetterWithInfo provides the


Looks like a truncated docblock.

Indeed it does.
Apparently, I also forgot to write the rest of the doc-comment on BackendGetInfo, too.

sergiosalvatore · 2025-11-18T15:57:38Z

galaxycache.go

+// The returned [BackendGetter] tries (but does not guarantee) to run only one
+// [Galaxy.Get] is called once for a given key across an entire set of peer
+// processes. Concurrent callers both in the local process and in
+// other processes receive copies of the answer once the original Get
+// completes.


Suggested change

// The returned [BackendGetter] tries (but does not guarantee) to run only one

// [Galaxy.Get] is called once for a given key across an entire set of peer

// processes. Concurrent callers both in the local process and in

// other processes receive copies of the answer once the original Get

// completes.

// The returned [BackendGetter] tries (but does not guarantee) to run only one

// instance of [Galaxy.Get] for a given key across an entire set of peer

// processes. Concurrent callers both in the local process and in

// other processes receive copies of the answer once the original Get

// completes.

wow, that wording was rather messed up.
It was a copy of NewGalaxy's doc-comment with a couple minor adjustments.

I've substantially reworded it. (your suggestion didn't seem sufficient)

sergiosalvatore · 2025-11-18T17:06:30Z

galaxycache.go

 }

 type GetInfo struct {
 	// TODO: include information about hit-level, backend fetches and any TTL (when implemented).


Does this comment still apply?

Good point.
I've removed TTL from the comment, moved it below Expiry and added some more doc-comments.

sergiosalvatore · 2025-11-18T17:17:03Z

jsoncodec/backend_getter.go

+// backendGetter is an adapter type that implements galaxycache.BackendGetter
+type backendGetterWithInfo[T any] func(ctx context.Context, key string) (T, galaxycache.BackendGetInfo, error)
+
+// Get populates dest with the value identified by key


Suggested change

// Get populates dest with the value identified by key

// Get populates dest with the value identified by `key`.

I might also backtick-quote the other instances of key so it's extra clear that we're referring to the argument.

Good call!
Done. (same in protocodec)

sergiosalvatore · 2025-11-18T17:23:12Z

lru/expiry/expiry_heap.go

+	xt.updateLocation(len(e.ents) - 1)
+}
+
+const sentinelOffset = math.MaxInt


Should this be math.MaxUInt given that offset is a uint?

Good question.

Unfortunately, we convert it to an int, so the value would be harder to compare after the conversion if we used math.MaxUint.

I've added a doc-comment.

sergiosalvatore · 2025-11-18T17:31:43Z

lru/typed_lru.go

+}

 // Add adds a value to the cache.
 func (c *TypedCache[K, V]) Add(key K, value V) {


Should this just call AddExpiring with a zeroed time.Time{}?

oh yeah, it can do that now.
My original implementation had split them over separate files with build tags so Add was available for all go versions, but I decided to just bump the go.mod to go 1.24, so I could merge it all back into one file.

Done.

Add expiry support for go 1.24+ with the help of weak pointers. Care has to be taken to make sure that Get doesn't return an expired value (and Add..., Remove... and friends are also configured for removing any expired entries before removing anything from the LRU stack).

Make it possible to remove entries that have been evicted from the cache by returning an opaque handle that can be passed to a Remove method.

Remove entries from the expiry tracker when they get removed.

Add a new BackendGetterWithInfo interface that allows a BackendGetter implementation to return an expiration time as well as functioning with the normal Codec mechanism for extracting the values. Plumb the BackendGetInfo with a new method on Universe, additional return values on Fetch & Peek, and update tests+http&grpc impls. Follow-up commits will update the grpc and http methods to propagate expirations.

Add expiry fields to both the Get and Peek responses so we can propagate expiration timestamps across process-boundaries.

Use a custom header format that includes nanoseconds so we don't lose fidelity. (note: we don't need to use the same format when parsing, and can instead use the stock http.ParseTime on the receiving side because Go's time.Parse automatically includes fractional seconds.)

Replace a deprecated call.

The tests on PR#318 flaked because of the log-after-test-completion race-detection. Avoid logging if the context is canceled. (this won't 100% eliminate the race, but it'll bring it down to a super-rare race)

Upgrade genproto within the compattest/peerv1.2 submodule to actually fix it, too.

sergiosalvatore

A couple more little nits...

sergiosalvatore · 2025-12-02T14:20:29Z

lru/expiry/expiry_heap.go

+	e.ents[i], e.ents[j] = e.ents[j], e.ents[i]
+}
+
+func (e *expiryHeap[T]) Push(x any) {


nit: we probably want a trivial doc comment...

Added.

Note that expiryHeap is unexported.

sergiosalvatore · 2025-12-02T14:20:45Z

lru/expiry/expiry_heap.go

+	e.ents = append(e.ents, xt)
+}
+
+func (e *expiryHeap[T]) Pop() any {


nit: doc comment for exported method.

added.
(same for EntryHandle)

sergiosalvatore · 2025-12-02T15:41:50Z

galaxycache.go

-	// TODO: include information about hit-level, backend fetches and any TTL (when implemented).
+	// If non-zero, Expiry provides an expiration time after which
+	// Galaxycache should not return this value (and should ideally evict
+	// it from the cache to prevent unexpired, but more recently-outched


nit:
s/outched/touched/

sergiosalvatore · 2025-12-02T15:43:23Z

peers.go


+// RemoteFetcherWithInfo is an extension of [RemoteFetcher], allowing
+// [RemoteFetcher] implementations to optionally provide [BackendGetInfo]. (the
+// zero-value is assumeed otherwise)


nit:
s/assumeed/assumed/

Base automatically changed from consistenthash_fnv_hash_wrapper to master June 9, 2025 16:17

dfinkel added 2 commits November 17, 2025 16:09

go.mod: bump to go 1.24.10

1c6fc35

go 1.25 is 3 months old and the 1.26 freeze is about to begin. Bump go.mod to 1.24 so we can use weak pointers without having to add new build tags.

go.mod: go mod tidy

636db46

dfinkel force-pushed the lru_ttl_support branch from 83dfd1e to 58ccee9 Compare November 17, 2025 21:26

dfinkel added 2 commits November 17, 2025 16:27

actions: upgrade to 1.24+1.25

8983b29

lru: add expiry subpackage to track expirations

83fc444

Leverage the container/heap stdlib package to manage a slice-backed heap for a first-pass.

dfinkel force-pushed the lru_ttl_support branch from 58ccee9 to 708c389 Compare November 17, 2025 21:28

dfinkel changed the title ~~Lru ttl support~~ TTL/Expiration setting & propagation Nov 17, 2025

dfinkel force-pushed the lru_ttl_support branch 2 times, most recently from ec7a68d to 91a715c Compare November 18, 2025 00:18

dfinkel marked this pull request as ready for review November 18, 2025 00:30

sergiosalvatore reviewed Nov 18, 2025

View reviewed changes

dfinkel added 13 commits November 18, 2025 14:11

lru/expiry: tests and add Remove method

ea33bb8

Make it possible to remove entries that have been evicted from the cache by returning an opaque handle that can be passed to a Remove method.

lru: call expiry tracker Remove upon eviction

6813d1b

Remove entries from the expiry tracker when they get removed.

galaxycachepb: add expiry fields

a6cc6db

Add expiry fields to both the Get and Peek responses so we can propagate expiration timestamps across process-boundaries.

test basic expiry handling & fix a few bugs

5f25eef

lint: replace grpc.Dial with grpc.NewClient

b0280e4

Replace a deprecated call.

grpc: implement expiration propagation

2c0b082

tests: cover peeking expiration

ee50982

jsoncodec: add BackendGetterWithInfo

5ed7ae0

protocodec: add BackendGetterWithInfo wrapper

2a31eca

compattest: add expiry propagation checks

2b339d2

dfinkel force-pushed the lru_ttl_support branch from 91a715c to fcffc54 Compare November 18, 2025 19:13

dfinkel added 2 commits November 18, 2025 14:37

compattest: log connstate changes if ctx not done

91f7ae5

The tests on PR#318 flaked because of the log-after-test-completion race-detection. Avoid logging if the context is canceled. (this won't 100% eliminate the race, but it'll bring it down to a super-rare race)

gomod: upgrade grpc to fix an ambiguous import

3163a8c

Upgrade genproto within the compattest/peerv1.2 submodule to actually fix it, too.

dfinkel force-pushed the lru_ttl_support branch from fcffc54 to 3163a8c Compare November 18, 2025 19:38

sergiosalvatore reviewed Dec 2, 2025

View reviewed changes

dfinkel added 2 commits December 3, 2025 14:17

lru/expiry: add some missing doc-comments

186266e

galaxycache: fix a couple typos

1165af2

sergiosalvatore approved these changes Dec 3, 2025

View reviewed changes

dfinkel merged commit 6459d31 into master Dec 3, 2025
8 checks passed

dfinkel deleted the lru_ttl_support branch December 3, 2025 21:03

	// Get populates dest with the value identified by key
	// Get populates dest with the value identified by `key`.

TTL/Expiration setting & propagation #75

TTL/Expiration setting & propagation #75

Uh oh!

Conversation

dfinkel commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sergiosalvatore left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sergiosalvatore left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dfinkel commented May 30, 2025 •

edited

Loading