-
Notifications
You must be signed in to change notification settings - Fork 293
Update to latest nostrdb #2121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Update to latest nostrdb #2121
Conversation
This broke a bunch of tests which I'm fixing now |
One thing I did here is add nostrdb's patch history to our local copy of nostrdb. This ensure we don't lose bisectability when updating nostrdb. I've also removed a lot of overlap between damus' C code and nostrdb, so most of the C code should be in nostrdb now. |
10bc4f4
to
c476272
Compare
1f3d846
to
f8931a6
Compare
8bc95f2
to
e56f0a5
Compare
@danieldaquino this is ready for testing! |
looks like there are a fix issues with block parsing... |
Fixed an iOS crash in the latest version of nostrdb. is a bus error (unaligned memory access) when trying to parse an empty json string when processing inner kind6 reposts (this is a new feature in this version of nostrdb) |
When trying to build from rust Signed-off-by: William Casarin <[email protected]>
Closes: damus-io/nostrdb#21 Signed-off-by: William Casarin <[email protected]>
No functional changes, just formatting cleanups Signed-off-by: William Casarin <[email protected]>
Add a helper for sorting search words from largest to smallest. This should help search performance. For example, let's say our search index is like so: "the pokemon is cool" the the the ... * 1000 Our root word search would have to start 1000 new recursive queries. By sorting by the largest word: pokemon pokemon pokemon ... * 10 We only have to do 10 recursive searches, assuming larger words are less common, which will likely be the case most of the time Signed-off-by: William Casarin <[email protected]>
Update fulltext search queries to include an optional filter. This can be used to narrow down the fulltext search. This is another step towards nip50 support in nostrdb. I noticed the code was exiting dubiously in certain situations... so we fix that as well. It's possible we were missing search results because of this. Signed-off-by: William Casarin <[email protected]>
This adds support for nip50 fulltext searches. This allows you to use the nostrdb query interface for executing fulltext searches instead of the typical `ndb_text_search` api. The benefits of this include a standardized query interface that also further filters on other fields in the filter. Changelog-Added: Add nip50 search filters and queries Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Add support for type KIND for bech32-encoded entities naddr and nevent as specified in NIP-19. Co-authored-by: kernelkind <[email protected]> Signed-off-by: William Casarin <[email protected]>
The last issue seems to be DM rendering |
The Address Sanitizer detected a heap buffer overflow during a memcpy operation in nostrdb.c associated with note parsing. It was found that not enough memory was being allocated to the buffer to support all the content parsing. Allocation size was increased to support the memory needed for the parsing operations. However, the new number was not carefully calculated as we will not run into this code path once we switch to the local relay model. Changelog-Fixed: Fixed memory error in nostrdb Signed-off-by: Daniel D’Aquino <[email protected]>
I wanted to not amend this since we've already applied it in the nostrdb-update branch on damus and I don't want to conflict Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
some binding dir stoppers, and configurator
doesn't need to create a copy of the id Signed-off-by: William Casarin <[email protected]>
This is still kind of broken until queries are switched over to nostrdb. Will do this next Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
Signed-off-by: William Casarin <[email protected]>
This commit introduces new interfaces for working with NostrDB from Swift, including `NostrFilter` conversion, subscription streaming via AsyncStreams and lookup/wait functions. No user-facing changes. Changelog-None Signed-off-by: Daniel D’Aquino <[email protected]>
Closes: #2885 Changelog-Changed: Use NostrDB for rendering note contents Signed-off-by: Daniel D’Aquino <[email protected]>
NostrDB relies on manual memory management, so it is a good idea to enable the address sanitizer on debug configurations, as it helps find memory-related issues on the app, which will allow us to identify memory issues and potential crashes earlier in the development process. Changelog-None Signed-off-by: Daniel D’Aquino <[email protected]>
This makes it easier to work from the command line when needed Changelog-None Signed-off-by: Daniel D’Aquino <[email protected]>
Some tests have been broken at some point during the nostrdb migration. Disable them for now and address them later (#3112) Signed-off-by: Daniel D’Aquino <[email protected]>
Currently NostrDB does not seem to handle encryption/decryption of DMs. Since NostrDB now controls the block parsing process and fetches note contents directly from the database, we have to add a specific condition that injects decrypted content directly to the ndb content parser. This is done in conjunction with some minor refactoring to `NdbBlocks` and associated structs, as in C those are separated between the content string and the offsets for each block, but in Swift this is more ergonomically represented as a standalone/self-containing object. No changelog entry is added because the previously broken version was never released to the public, and therefore this fix produces no user-facing changes compared to the last released version. Changelog-None Closes: #3106 Signed-off-by: Daniel D’Aquino <[email protected]>
771ae04
to
9dcfe3c
Compare
This branch updates nostrdb to the latest version
Things done and left to do:
- [ ] Switch to local relay for querying and subscriptionssaving this for laterWhat's new in this version: