Storage Decisions

[jpronans]

Afternoon,
Is there a recommended storage layout for a node? I.e. Sizes of partitions and fileystem requirements.

[donalhunt]

Hi @jpronans!

There doesn't seem to be a good recommendation but it does seem like we need to come up with some "standard" for OSM Ireland / the dev*.openstreetmap.ie server(s).

This is the current layout:

Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1       9.6G  5.5G  4.1G  57% /
/dev/vda15      105M  6.3M   99M   7% /boot/efi
/dev/vdc        103G   44G   54G  45% /mnt/vdc
/dev/vdd         48G  7.9G   37G  18% /mnt/vdd
/dev/vde        503G   67G  411G  14% /mnt/vdh
/dev/vdf         47G  3.7G   41G   9% /var/lib/docker

/dev/vdc (mounted as /dev/vdc)seems to be a /data mountpoint dating from ~ Nov 2019 (last write is Nov 2019).
/dev/vdd (mounted as /dev/vdd) seems to be a /var mountpoint dating from 2018-2019 (last write is Dec 2019).
/dev/vde (mounted as /dev/vdh) seems to be a /data mountpoint dating from this year (writes from 2021).
/dev/vdf (mounted as /var/lib/docker) seems to setup for docker usage (setup in March 2021 based on dates).

Summary of current devices in use

UUID	filesystem	mountpoint	label
	/dev/vda1	/	cloudimg-rootfs
	/dev/vda15	/boot/efi	UEFI
3f5b92c9-6170-47aa-b8f7-21ca607a46a9	/dev/vdc	/mnt/vdc
ebac04d1-7d75-45fa-9406-e29b28111192	/dev/vdd	/mnt/vdd
5ce43ee9-5d95-49dd-a794-3f0ce1e546df	/dev/vde	/mnt/vdh
db99905e-a480-4314-80c6-ac36f90e2029	/dev/vdf	/var/lib/docker

Suggested approach

/ should be more than enough with 10G of storage on any server. Could tune that down at a later date.
/docker probably needs a reasonable amount of space for runtime purposes (image storage, etc). 40G seems good right now.
/data is where any long-lived data should be stored (would be mounted into docker images where appropriate). 100G is a good first guess but will likely be lower once stuff is better organised and we have estimates for different things.

[jpronans]

On 9/26/21 5:59 PM, Donal Hunt wrote: Hi @jpronans <https://github.com/jpronans>! There doesn't seem to be a good recommendation but it does seem like we need to come up with some "standard" for OSM Ireland / the dev*.openstreetmap.ie server(s).

I completely agree... it has been very organic up to know done on a reactive rather than pro-active basis, so it is a bit of a mess.

This is the current layout: |Filesystem Size Used Avail Use% Mounted on /dev/vda1 9.6G 5.5G 4.1G 57% / /dev/vda15 105M 6.3M 99M 7% /boot/efi /dev/vdc 103G 44G 54G 45% /mnt/vdc /dev/vdd 48G 7.9G 37G 18% /mnt/vdd /dev/vde 503G 67G 411G 14% /mnt/vdh /dev/vdf 47G 3.7G 41G 9% /var/lib/docker | |/dev/vdc| (mounted as |/dev/vdc|)seems to be a |/data| mountpoint dating from ~ Nov 2019 (last write is Nov 2019). |/dev/vdd| (mounted as |/dev/vdd|) seems to be a |/var| mountpoint dating from 2018-2019 (last write is Dec 2019). |/dev/vde| (mounted as |/dev/vdh|) seems to be a |/data| mountpoint dating from this year (writes from 2021). |/dev/vdf| (mounted as |/var/lib/docker|) seems to setup for docker usage (setup in March 2021 based on dates).

Yes, they are data from the old server I've just kept on volumes in case they were needed.

Summary of current devices in use UUID filesystem mountpoint label /dev/vda1 / cloudimg-rootfs /dev/vda15 /boot/efi UEFI 3f5b92c9-6170-47aa-b8f7-21ca607a46a9 /dev/vdc /mnt/vdc ebac04d1-7d75-45fa-9406-e29b28111192 /dev/vdd /mnt/vdd 5ce43ee9-5d95-49dd-a794-3f0ce1e546df /dev/vde /mnt/vdh db99905e-a480-4314-80c6-ac36f90e2029 /dev/vdf /var/lib/docker Suggested approach |/| should be more than enough with 10G of storage on any server. Could tune that down at a later date.

I'd start with 20GB myself.

|/docker| probably needs a reasonable amount of space for runtime purposes (image storage, etc). 40G seems good right now.

So lets say 50GB

|/data| is where any long-lived data should be stored (would be mounted into docker images where appropriate). 100G is a good first guess but will likely be lower once stuff is better organised and we have estimates for different things

So lets go with 200GB then. How does that sound? John

…

. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAE64FTKSNWKR2KJ7Z2GUPDUD5GO5ANCNFSM4M2OMCYA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.