Improve cookiecutter and demo-site command

[Tiago-Salles]

In quick words, this PR includes:

• Add org_dockerhub, nginx_image_name, nginx_image_tag, and elasticsearch_image_name parameters to generate a site factory.
• Select database engine to run a site.
• Generate a new site selecting the second language.
• Activate a site with an optional question to select the supported languages.
• Improve create_demo_site command script to support multilingual generation.

Purpose

We at NAU identified the necessity to have more flexibility in terms of architecture using Richie.

For our instance we have been maintaining some changes to meet our needs, that we understand valuable to have upstream. Our instance runs with MySQL and the docker images come from our dockerhub.

This PR addresses these requirements, and provide more flexibility in terms of architecture.

How does it works

Since this PR deals with the site factory as well, it is necessary to generate a new site factory and after generate a new site.

Generate the site factory

To run the site-factory generation according to this proposal, execute this command in the root of the project:

docker run --rm -it -u $(id -u):$(id -g) -v $PWD:/app fundocker/cookiecutter gh:Tiago-Salles/richie --directory cookiecutter --checkout Tiago-Salles/issues/274-upstream-cookiecutter

It prompts four questions besides the current question organization:

• org_dockerhub: Which will the main dockerhub the project will pull the images, which if not provided it will use the default fundocker.
• nginx_image_name: the nginx image name, which if not provided it will use the default fundocker/openshift-nginx.
• nginx_image_tag: the nginx image tag, which if not provided it will use the default 1.13.
• elasticsearch_image_name: the elasticsearch image name, which if not provided it will use the default fundocker/openshift-elasticsearch:6.6.2.

Generate the new site

In the root of the generated site factory, it is necessary to run make add-site providing the standard information and the second language this site will support.

In order to have the make demo-site according to the changes made here, replace in the base.txt requirements file the richie package with git+https://github.com/Tiago-Salles/richie@Tiago-Salles/issues/274-upstream-cookiecutter#egg=richie.

Activate the new site

In the root of the site factory it is necessary to run bin/activate so it is possible to select the database. It is worth mentioning that this functionality also provides the possibility of switching the database engine for a running site. Select as well the languages that this site will support.

Fill in the new site

Now, with the richie package replaced with git+https://github.com/Tiago-Salles/richie@Tiago-Salles/issues/274-upstream-cookiecutter#egg=richie in the base.txt of the new site, it is necessary to run make bootstrap and then make demo-site.

What happens in each step

Generate the site factory

All the answers will be used to the define infrastructure resources. These answers will be set on the files that manage the site-factory and site, such as Makefile, .circleci folder, site settings.py and the other similars ones.

Generate the new site

This step adds to the questions, the second language this site will support.

Activate the new site

This step is focused only in the site. That is, all the content generated here will not impact at the site factory resources.

Besides the second language that already was set using cookiecutter, it is possible here to set more languages in the environment. This step is optinal, if none insterests exist on this, it can be skipped.

Here it sets in the .env the variables:

• RICHIE_SITE: The site name.

• ACTIVATED_DB: The site database engine which will be called in the docker-compose.yml file, being included as following

    include:
        - docker-compose-${ACTIVATED_DB}.yml
    services: ...
  

correspoding to the file in the root of the site factory

Where in each file is set their business logics as well their dependencies.

• DB_PORT: Automatically set in according to the ACTIVATED_DB variables, in instance of being mysql its value will be 3306. If the interest is to use another port, feel free to change the value setting a new one in the .env file as DB_PORT={new_port}.

• LANGUAGES: The site supported languages. It will be called in the settings.py as following

    LANGUAGES = os.getenv("LANGUAGES") or [("en", _("English")), ("fr", _("French"))]
  
    if isinstance(LANGUAGES, str):
        try:
            LANGUAGES = ast.literal_eval(LANGUAGES)
            LANGUAGES = [(code, _(name)) for code, name in LANGUAGES]
        except:
            LANGUAGES = [("en", _("English")), ("fr", _("French"))]
  

Important Observations

It is important to understand that only these mentioned steps are not enough, if on the activate site process was selected the Korean language as one of the languages the site will support, it to must have the ko-KR.json file in the ./sites/{site_name}/src/frontend/i18n/locales/, otherwise the frontend will not provide the possibility to use this language.

By default the backend already provides this language content, but it is worth to check the corresponding language configuration is there as expected, by going to ./sites/{site_name}/src/backend/locale and verifying.

As already mentioned, it is important to replace the richie package with git+https://github.com/Tiago-Salles/richie@Tiago-Salles/issues/274-upstream-cookiecutter#egg=richie in the base.txt of the new site to have all this new features to work.

[igobranco]

@Tiago-Salles please include instructions to testing on the PR description.

[igobranco]

I think the following command execute the cookiecutter of run your PR:

docker run --rm -it -u $(id -u):$(id -g) -v $PWD:/app fundocker/cookiecutter gh:Tiago-Salles/richie --directory cookiecutter --checkout Tiago-Salles/issues/274-upstream-cookiecutter

[Tiago-Salles]

I think the following command execute the cookiecutter of run your PR:

docker run --rm -it -u $(id -u):$(id -g) -v $PWD:/app fundocker/cookiecutter gh:Tiago-Salles/richie --directory cookiecutter --checkout Tiago-Salles/issues/274-upstream-cookiecutter

Exactly @igobranco. I used a difference approach to develop, just stepping back and pointing to the directory of the project containing the changes, but this solution you provided is better, as it can be run from any branch. Thank you, I added to the PR description.

[igobranco]

@Tiago-Salles Please include the "languages" has a cookiecutter question.
So this block will also be generated from those "languages" answer.

richie/cookiecutter/{{cookiecutter.organization}}-richie-site-factory/template/{{cookiecutter.site}}/src/backend/{{cookiecutter.site}}/settings.py

Lines 473 to 501 in 5deda56

	LANGUAGES = (("en", _("English")), ("fr", _("French")))
	# - Django CMS
	CMS_LANGUAGES = {
	"default": {
	"public": True,
	"hide_untranslated": False,
	"redirect_on_fallback": True,
	"fallbacks": ["en", "fr"],
	},
	1: [
	{
	"public": True,
	"code": "en",
	"hide_untranslated": False,
	"name": _("English"),
	"fallbacks": ["fr"],
	"redirect_on_fallback": False,
	},
	{
	"public": True,
	"code": "fr",
	"hide_untranslated": False,
	"name": _("French"),
	"fallbacks": ["en"],
	"redirect_on_fallback": False,
	},
	],
	}

[igobranco]

@Tiago-Salles review the commit message:

From:
🧱(infra) select databases to run sites

To:
🧱(cookiecutter) select databases to run sites

Personally, I don't like the logo... why not replace with 🚩? You are just introducing flags... kind of... IMO questions of cookiecutter can be flags!

[igobranco]

@Tiago-Salles

After creating the factory site and adding a site.

The cookie cutter generated site factory Makefile isn't showing the help correctly:

$ make help
Makefile                       add a new site to the site factory
Makefile                       install development dependencies
Makefile                       build all containers
Makefile                       build front-end application
Makefile                       build Sass files to css
Makefile                       build Sass files to css (production mode)
Makefile                       build ts(x) files to js
Makefile                       build ts(x) files to js (production mode)
Makefile                       perform django checks

cat .env
RICHIE_SITE=asa
ACTIVATED_DB=postgresql
DB_PORT=3306

When running the site it stops on the collectstatic.

make run

Error:

....
 => CACHED [app core  9/10] COPY ./docker/files/usr/local/etc/gunicorn/app.py /usr/local/etc/gunicorn/app.py                                       0.0s
 => CACHED [app core 10/10] RUN chmod g=u /etc/passwd                                                                                              0.0s
 => CACHED [app collector 1/3] RUN apt-get update &&     apt-get install -y     rdfind &&     rm -rf /var/lib/apt/lists/*                          0.0s
 => ERROR [app collector 2/3] RUN python manage.py collectstatic --noinput                                                                         0.2s
------
 > [app collector 2/3] RUN python manage.py collectstatic --noinput:
0.178 Unknown command: 'collectstatic'
0.178 Type 'manage.py help' for usage.
------
failed to solve: process "/bin/sh -c python manage.py collectstatic --noinput" did not complete successfully: exit code: 1
make: *** [Makefile:104: run] Error 1
....

[Tiago-Salles]

We are facing in this branch the same problem as highlighted in the collectstatic command is not being recognized.

[igobranco]

@Tiago-Salles
Fix the duplication of installing richie on base.txt:
richie==3.0.0

After you change this, IMO is good enough to be merged.

[Tiago-Salles]

@igobranco I have made the changes you identified necessary, we are good to pass this on. @kernicPanel Please, could you review this PR?

[kernicPanel]

Sorry for the delay.
LGTM 👍

[igobranco]

@kernicPanel
@Tiago-Salles is now on 3 week holidays, so we can wait until he is available to rebase, fix the conflict related to new Richie release, change the CHANGELOG.md line to the correct location, etc.
This PR is important but not critical like the one that you merged yesterday! Sorry but yesterday I were resolving other internal issues...