_purge API for only old revisions

[H--o-l]

Hi!

I have one question about the _purge API.

I run CouchDB v2.3.1.
I have one database with one document inside it.

This document has 10 revisions, each of them results of a conflict resolution.

$ curl -sSf -u $CRED localhost:5984/database/document?revs_info=true | jq -M
{
  "_id": "document",
  "_rev": "10-46374dfdbd4a42679e553da796ddfa05",
  "data": "8-toto",
  "_revs_info": [
    {
      "rev": "10-46374dfdbd4a42679e553da796ddfa05",
      "status": "available"
    },
    {
      "rev": "9-be1dbaf872f4ab04bbd76ec87b559a03",
      "status": "available"
    },
    {
      "rev": "8-ae84b773e5fda553724150c9929ac110",
  ...

I try the following:

$ curl -sSf -u $CRED localhost:5984/database/_purge \
    -H 'Content-Type: application/json' \
    -d '{"document": ["8-ae84b773e5fda553724150c9929ac110"]}'
{"purge_seq":null,"purged":{"document":[]}}

And it doesn't purge anything.
The only thing I can purge is the last revision 10-46374dfdbd4a42679e553da796ddfa05.

Do you have an idea how I could purge the old revisions, without deleting the
last one?

You may ask why I want to do that.
I have a performance issue (memory leak) with PouchDB when I tried to replicate
document with a lot of old resolved conflicts (more than 1000 old resolved conflicts
and my Chrome crashes).
It's another story for another time but if someone already experience this with PouchDB,
I'd be happy to hear your experience.

[wohali]

Hey @H--o-l , thanks for asking.

https://docs.couchdb.org/en/latest/api/database/misc.html#db-purge has a really good explanation of why this won't work, as you've explained it.

If your revision tree has branches, you can purge the branches that aren't direct ancestors of the "winning revision."

If your revision tree is a straight line, as in your example, you can't purge the preceding versions.

If, after reading the link, you still have questions, let me know and I'll try and explain further.

[wohali]

Also worth mentioning: if your revision tree is now a straight line, you can try forcibly compacting the database instead of purging. That will remove previous revisions in the straight-line tree.

[H--o-l]

Hi @wohali thanks for the really quick answer!

https://docs.couchdb.org/en/latest/api/database/misc.html#db-purge has a really good explanation of why this won't work, as you've explained it.

You are right, the documentation explains it well that it will purge only leaf revisions.

If, after reading the link, you still have questions, let me know and I'll try and explain further.
if your revision tree is now a straight line, you can try forcibly compacting the database instead of purging

So the solution for me is the compaction.
But I played more with it, and I have issue when there is "winning revision" in the history tree.

My issue is showed with the following example:
I have a tree of revisions, with a conflict resolved each three revisions:

$ curl -sSf -u $CRED localhost:5984/database/document?revs_info=true | jq -M
{
  "_id": "document",
  "_rev": "30-665ac1f4b5b595480e641810821b657e",
  "data": "f-toto",
  "_revs_info": [
    {
      "rev": "30-665ac1f4b5b595480e641810821b657e",
      "status": "available"
    },
    {
      "rev": "29-2f084e5446c287fd716abf23b9bc6dac",
      "status": "missing"
    },
    {
      "rev": "28-a63f2e181cd879fb96d0ec2f2d17efd3",  # <=== from a resolve conflict
      "status": "missing"
    },
    {
      "rev": "27-13235ace4046238f2696fb374a3b0dba",
      "status": "missing"
    },
    {
      "rev": "26-6e041fa7218b6527711dee61ea2d4e95",
      "status": "missing"
    },
    {
      "rev": "25-7d609e3ee21e76fceb81e6113dbf7e92",   # <=== from a resolve conflict
      "status": "missing"
    },
  ...

(here is the code I use)

If I do the following command, the 30 revisions metadata are kept:

$ curl -X PUT "localhost:5984/database/_revs_limit" -d "10"
$ curl -X POST localhost:5984/database/_compact -H 'Content-Type: application/json'

But If I do the following command, only the last 5 revisions metadata are kept:

$ curl -X PUT "localhost:5984/database/_revs_limit" -d "5"
$ curl -X POST localhost:5984/database/_compact -H 'Content-Type: application/json'

The only difference is the value of revs_limit, from 10 to 5.
The value 10 doesn't seems to work.

Do you know if this behavior is expected, and what may explains it?

[wohali]

If you mess with _revs_limit you can potentially break replication, if the common ancestor is lost - as you have seen.

Don't do that.

[H--o-l]

Hey @wohali thanks for your answer again!
And thanks for taking the time to answer me, it's really kind and I value your time.

You indicate the discussion as answered, but my original question was:

Do you have an idea how I could purge the old revisions, without deleting the
last one?

And in my case, when I have a lot of old revision, which are the result of conflicts resolution, I still can not delete them, either by _purge or compaction.
(I can reword the title of the discussion perhaps?)

If you mess with _revs_limit you can potentially break replication, if the common ancestor is lost - as you have seen.

In my case it's exactly what I want, I want to remove all the ancestors after a certain number of revisions, because their metadata are causing huge performance issue for PouchDB.
I think that the _revs_limit could be used for that, but I missed this option at first because his behavior seems awkward (sometime it works, sometime it doesn't, as show above).

Does anyone have tried to play with these options, and an history tree full of old conflicts?

[H--o-l]

Hey!

I find the answers I was looking for.
If anyone ever asks themselves the same questions:

To clean old revisions when you have resolved conflicts in your tree:

You first need to purge the "losing revisions".
You can get them with the following (see the ?meta=true):

$ curl -sSf "localhost:5984/database/document?meta=true"
{
  "_id": "document",
  "_rev": "30-a3dbf287d0d1ece7bbe8b784db603dc2",
...
  ],
  "_deleted_conflicts": [
    "29-3168b4b8cd02c25c6a0f6c34539b631d",
    "26-05786d423bfe1cbf82e3d6c73bf2e7c9",
    "23-34e3dfdc994ad000c64a21cd30049cb3",
    "20-5d9beef9eeb48ad773b00f04b9d14e38",
    "17-a9f7e63943620255a66e9629466c7e66",
    "14-740198ac11897c6b9f2e9ff768561273",
    "11-444580622bdec6165a6ba4dc4626dff1",
    "8-1191b3419111d593d07aabe32bdd6383",
    "5-a3beb663f5c182414e09b502f6de2379"
  ]
}

Then purge them:

$ curl -sSf localhost:5984/database/_purge \
   -H 'Content-Type: application/json' \
   -d '
{
  "document": [
    "29-3168b4b8cd02c25c6a0f6c34539b631d",
    "26-05786d423bfe1cbf82e3d6c73bf2e7c9",
...
  ]
}

Will have have a straight line history, and the _revs_limit and compaction will behave as you might expect.

RAM issue with PouchDB:

Purge the "losing revisions" as above.
When the history will be a straight line, the issue will disappear, you don't need to mess with _revs_limit or the compaction.

Of course don't do commands above if you need the document history (conflicts could appear).

Thanks @wohali for your help on the subject. Also thanks @kocolosk for your comment here in 2018, it helps me understand what to do.

[jiangphcn]

Glad to see that _purge helps.