Elasticsearch populate starts before clone finishes

[jhanarato]

With all volumes removed and the checkout directory deleted, we get this error:

make[1]: Entering directory '/home/jr/Code/elastic-bilara'
Populating Elasticsearch with data..
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/app/search/search.py", line 37, in __init__
    self._populate_index(settings.ES_INDEX, settings.ES_SEGMENTS_INDEX)
  File "/app/search/search.py", line 58, in _populate_index
    self._process_data(index, segments_index)
  File "/app/search/search.py", line 474, in _process_data
    for data in self._yield_data(paths):
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/search/search.py", line 123, in _yield_data
    for file_path in iterator:
                     ^^^^^^^^
  File "/app/search/utils.py", line 53, in yield_file_path
    for file in os.listdir(work_dir):
                ^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/app/checkouts/unpublished'
make[1]: *** [Makefile:12: populate_elastic] Error 1
make[1]: Leaving directory '/home/jr/Code/elastic-bilara'
make: *** [Makefile:24: build] Error 2

It seems that the second git clone command in entrypoint.sh does not complete before the application is launched.

[jhanarato]

Nope, it's clear from the error message:

make[1]: *** [Makefile:12: populate_elastic] Error 1

We have a sleep of 60 seconds:

elastic-bilara/Makefile

Lines 20 to 24 in 32dad15

	build:
	@$(COMPOSE) up --build -d
	@make copy_cert
	@sleep 60
	@make populate_elastic

We might need to wait longer, or signal that the process is complete somehow.

[jhanarato]

It's worth noting that even if the population doesn't take place when it should, one of our Celery workers will do the job eventually. Hard to know if it is a complete job though.

[jhanarato]

I can get a clean build with:

docker compose down -v
docker compose up --build -d
make copy_cert
make populate_elastic