Skip to content

Commit 82320e5

Browse files
whiklojwhikloj
authored
Move all microservices to Symfony (#133)
* Update Houdini to 4.4 and convert Homarus to Symfony * Move hypercube to Symfony 4.4 * make crayfish-commons config generic * Add log files for Hypercube * Move Milliner to Symfony 4.4 * Move Recast to Symfony 4.4 * Update crayfish-commons version * Add upgrade instructions
1 parent 09a62a4 commit 82320e5

198 files changed

Lines changed: 21039 additions & 10997 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/build-3.x.yml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# This is a basic workflow to help you get started with Actions
2+
3+
name: CI
4+
5+
# Controls when the action will run.
6+
on:
7+
# Triggers the workflow on push or pull request events but only for the 7.x branch
8+
push:
9+
branches: [ 3.x ]
10+
pull_request:
11+
branches: [ 3.x ]
12+
13+
# Allows you to run this workflow manually from the Actions tab
14+
workflow_dispatch:
15+
16+
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
17+
jobs:
18+
# This workflow contains a single job called "build"
19+
build:
20+
# The type of runner that the job will run on
21+
runs-on: ubuntu-latest
22+
strategy:
23+
matrix:
24+
php-versions: ["7.3", "7.4"]
25+
26+
name: PHP ${{ matrix.php-versions }}
27+
28+
# Steps represent a sequence of tasks that will be executed as part of the job
29+
steps:
30+
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
31+
- name: Checkout code
32+
uses: actions/checkout@v2
33+
with:
34+
path: build_dir
35+
36+
- name: Checkout islandora_ci
37+
uses: actions/checkout@v2
38+
with:
39+
repository: islandora/islandora_ci
40+
ref: github-actions
41+
path: islandora_ci
42+
43+
- name: Setup PHP
44+
uses: shivammathur/setup-php@v2
45+
with:
46+
php-version: ${{ matrix.php-versions }}
47+
tools: composer:v2
48+
49+
- name: Set environment variables
50+
run: |
51+
echo "SCRIPT_DIR=$GITHUB_WORKSPACE/islandora_ci" >> $GITHUB_ENV
52+
53+
- name: Cache Composer dependencies
54+
uses: actions/cache@v2
55+
with:
56+
path: /tmp/composer-cache
57+
key: ${{ runner.os }}-${{ hashFiles('**/composer.lock') }}
58+
59+
- name: composer install
60+
run: |
61+
cd $GITHUB_WORKSPACE/build_dir
62+
for D in */; do (cd $D; composer install) done
63+
64+
- name: line endings
65+
run: $SCRIPT_DIR/line_endings.sh $GITHUB_WORKSPACE
66+
67+
- name: test scripts
68+
run: |
69+
cd $GITHUB_WORKSPACE/build_dir
70+
.scripts/tester
71+
72+
- name: codecov
73+
uses: codecov/codecov-action@v1
74+

‎Homarus/.env‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# In all environments, the following files are loaded if they exist,
2+
# the latter taking precedence over the former:
3+
#
4+
# * .env contains default values for the environment variables needed by the app
5+
# * .env.local uncommitted file with local overrides
6+
# * .env.$APP_ENV committed environment-specific defaults
7+
# * .env.$APP_ENV.local uncommitted environment-specific overrides
8+
#
9+
# Real environment variables win over .env files.
10+
#
11+
# DO NOT DEFINE PRODUCTION SECRETS IN THIS FILE NOR IN ANY OTHER COMMITTED FILES.
12+
#
13+
# Run "composer dump-env prod" to compile .env files for production use (requires symfony/flex >=1.2).
14+
# https://symfony.com/doc/current/best_practices.html#use-environment-variables-for-infrastructure-configuration
15+
16+
###> symfony/framework-bundle ###
17+
APP_ENV=dev
18+
APP_SECRET=2debbf0f3bc4a9484b577b8952dc3477
19+
#TRUSTED_PROXIES=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
20+
#TRUSTED_HOSTS='^(localhost|example\.com)$'
21+
###< symfony/framework-bundle ###

‎Homarus/.env.test‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# define your env variables for the test env here
2+
KERNEL_CLASS='App\Kernel'
3+
APP_SECRET='$ecretf0rt3st'
4+
SYMFONY_DEPRECATIONS_HELPER=999999
5+
PANTHER_APP_ENV=panther
6+
PANTHER_ERROR_SCREENSHOT_DIR=./var/error-screenshots

‎Homarus/README.md‎

Lines changed: 40 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,32 +1,27 @@
1-
![homarus](https://user-images.githubusercontent.com/2371345/48797524-c8c14300-ecd8-11e8-907d-9628fb6afacc.png)
2-
# Homarus
1+
![homarus](https://user-images.githubusercontent.com/2371345/48797524-c8c14300-ecd8-11e8-907d-9628fb6afacc.png)
2+
# Homarus
33

44
## Introduction
55

66
[FFmpeg](https://www.ffmpeg.org/) as a microservice.
77

88
## Installation
9-
- Install `ffmpeg`. On Ubuntu, this can be done with `sudo apt-get install ffmpeg`.
9+
- Install `ffmpeg`. On Ubuntu, this can be done with `sudo apt-get install ffmpeg`.
1010
- Clone this repository somewhere in your web root (example: `/var/www/html/Crayfish/Homarus`).
11-
- Copy `/var/www/html/Crayfish/Homarus/cfg/config.default.yml` to `/var/www/html/Crayfish/Homarus/cfg/config.yml`
12-
- Copy `/var/www/html/Crayfish/Hypercube/syn-settings.xml` to `/var/www/html/Crayfish/Homarus/syn-settings.xml`
1311
- Install `composer`. [Install instructions here.][4]
1412
- `$ cd /path/to/Homarus` and run `$ composer install`
15-
- Then either
16-
- For production, configure your web server appropriately (e.g. add a VirtualHost for Homarus in Apache) OR
17-
- For development, run the PHP built-in web server `$ php -S localhost:8888 -t src` from Homarus root.
18-
13+
- Configure your web server appropriately (e.g. add a VirtualHost for Homarus in Apache)
1914

2015
### Apache2
2116

2217
To use Homarus with Apache you need to configure your Virtualhost with a few options:
2318
- Redirect all requests to the Homarus index.php file
24-
- Make sure Hypercube has access to Authorization headers
19+
- Make sure Homarus has access to Authorization headers
2520

2621
Here is an example configuration for Apache 2.4:
2722
```apache
28-
Alias "/homarus" "/var/www/html/Crayfish/Homarus/src"
29-
<Directory "/var/www/html/Crayfish/Homarus/src">
23+
Alias "/homarus" "/var/www/html/Crayfish/Homarus/public"
24+
<Directory "/var/www/html/Crayfish/Homarus/public">
3025
FallbackResource /homarus/index.php
3126
Require all granted
3227
DirectoryIndex index.php
@@ -36,14 +31,44 @@ Alias "/homarus" "/var/www/html/Crayfish/Homarus/src"
3631

3732
This will put the Homarus at the /homarus endpoint on the web server.
3833

34+
## Upgrading
35+
36+
Steps for upgrading Homarus can be found in [UPGRADE.md](UPGRADE.md)
37+
3938
## Configuration
4039

41-
If your ffmpeg installation is not on your path, then you can configure homarus to use a specific executable by editing `executable` entry in [config.yaml](./cfg/config.example.yaml).
40+
Symfony uses `.dotenv` to set environment variables. You can check the [.env](./.env) in the root of the Homarus directory.
41+
To alter any settings, create a file called `.env.local` to store your specific changes. You can also set an actual environment
42+
variable.
43+
44+
For production use make sure to set the add `APP_ENV=prod` environment variable.
45+
46+
If your `ffmpeg` installation is not on your path, then you can configure homarus to use a specific executable by editing
47+
the `app.executable` parameter in [`/path/to/Homarus/config/services.yaml`](./config/services.yaml).
48+
49+
You also need to set your Fedora Base Url to allow the Fedora Resource to be pulled in automatically.
50+
This is done in the `/path/to/Homarus/config/packages/crayfish_commons.yaml`.
51+
52+
Also in the `/path/to/Homarus/config/packages/crayfish_commons.yaml` file you can point to the location of your `syn-settings.xml`.
53+
If you don't have a `syn-settings.xml` look at the [Syn](http://github.com/Islandora/Syn) documentation.
4254

43-
You also will need to set the `fedora base url` entry to point to your Fedora installation.
55+
### Logging
56+
57+
To change your log settings, edit the `/path/to/Homarus/config/packages/monolog.yaml` file.
58+
59+
You can also copy the file into one of the `/path/to/Homarus/config/packages/<environment>` directories.
60+
Where `<environment>` is `dev`, `test`, or `prod` based on the `APP_ENV` variable (see above). The files in the specific
61+
environment directory will take precedence over those in the `/path/to/Homarus/config/packages` directory.
62+
63+
The location specified in the configuration file for the log must be writable by the web server.
64+
65+
### Disabling Syn
66+
67+
There are instructions in the `/path/to/Homarus/config/packages/security.yaml` file describing what to change and what lines
68+
to comment out to disable Syn.
4469

4570
## Usage
46-
This will return the an AVI file for the test video file in Fedora.
71+
This will return the an AVI file for the test video file in Fedora.
4772
```
4873
curl -H "Authorization: Bearer islandora" -H "Accept: video/x-msvideo" -H "Apix-Ldp-Resource:http://localhost:8080/fcrepo/rest/testvideo" http://localhost:8000/homarus/convert --output output.avi
4974
```
@@ -60,8 +85,6 @@ If you would like to contribute, please get involved by attending our weekly [Te
6085

6186
If you would like to contribute code to the project, you need to be covered by an Islandora Foundation [Contributor License Agreement](http://islandora.ca/sites/default/files/islandora_cla.pdf) or [Corporate Contributor License Agreement](http://islandora.ca/sites/default/files/islandora_ccla.pdf). Please see the [Contributors](http://islandora.ca/resources/contributors) pages on Islandora.ca for more information.
6287

63-
We recommend using the [islandora-playbook](https://github.com/Islandora-Devops/islandora-playbook) to get started.
64-
6588
## License
6689

6790
[MIT](https://opensource.org/licenses/MIT)

‎Homarus/UPGRADE.md‎

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
This document guides you through the process of upgrading Homarus. First, check if a section named "Upgrade to x.x.x" exists, with x.x.x being the version you are planning to upgrade to.
2+
3+
## Upgrade to 3.0.0
4+
5+
Homarus (and all of Crayfish) adheres to [semantic versioning](https://semver.org), which makes a distinction between "major", "minor", and "patch" versions. The upgrade path will be different depending on which previous version from which you are migrating.
6+
7+
### Upgrade from version 2.x.x
8+
9+
Homarus has switched from a Silex application to a Symfony application. This does not require much in code changes, but does use a different file layout.
10+
11+
Previously your configuration file would be located in the `/path/to/Homarus/config` directory and be called `config.yaml`.
12+
13+
The configuration from this file will now be located several locations documented below.
14+
15+
#### FFMpeg executable location
16+
Old location `/path/to/Homarus/config/config.yaml`
17+
18+
```
19+
---
20+
homarus:
21+
# path to the ffmpeg executable
22+
executable: ffmpeg
23+
```
24+
25+
The `executable` variable is now located in `/path/to/Homarus/config/services.yaml` and appears in the `parameters`
26+
27+
```
28+
parameters:
29+
app.executable: ffmpeg
30+
```
31+
32+
#### Mimetypes and mime\_to_format
33+
Old location `/path/to/Homarus/config/config.yaml`
34+
35+
```
36+
---
37+
homarus:
38+
...
39+
mime_types:
40+
valid:
41+
- video/mp4
42+
- video/x-msvideo
43+
- video/ogg
44+
- audio/x-wav
45+
- audio/mpeg
46+
- audio/aac
47+
- image/jpeg
48+
- image/png
49+
default: video/mp4
50+
mime_to_format:
51+
valid:
52+
- video/mp4_mp4
53+
- video/x-msvideo_avi
54+
- video/ogg_ogg
55+
- audio/x-wav_wav
56+
- audio/mpeg_mp3
57+
- audio/aac_m4a
58+
- image/jpeg_image2pipe
59+
- image/png_image2pipe
60+
default: mp4
61+
```
62+
63+
The two lists (`mime_types.valid` and `mime_to_format.valid`) have been combined into a single list. The new variable is now located in `/path/to/Homarus/config/services.yaml` and appears in the `parameters`
64+
65+
The `mime_to_format` variable in version 2.x.x was the combination of the mimetype, the underscore character ( _ ) and the FFMpeg format (ie. `video/mp4_mp4`). In version 3.0.0 we create a list with keys `mimetype` and `format`.
66+
67+
```
68+
parameters:
69+
...
70+
app.formats.valid:
71+
- mimetype: video/mp4
72+
format: mp4
73+
- mimetype: video/x-msvideo
74+
format: avi
75+
- mimetype: video/ogg
76+
format: ogg
77+
- mimetype: audio/x-wav
78+
format: wav
79+
- mimetype: audio/mpeg
80+
format: mp3
81+
- mimetype: audio/aac
82+
format: m4a
83+
- mimetype: image/jpeg
84+
format: image2pipe
85+
- mimetype: image/png
86+
format: image2pipe
87+
```
88+
89+
The two `default` variables (`mime_type.default`, `mime_to_format.default`) have been combined and moved to the `app.formats.defaults` variable
90+
91+
```
92+
parameters:
93+
...
94+
app.formats.defaults:
95+
mimetype: video/mp4
96+
format: mp4
97+
```
98+
99+
#### Fedora Resource
100+
Old location `/path/to/Homarus/config/config.yaml`
101+
102+
```
103+
...
104+
fedora_resource:
105+
base_url: http://localhost:8080/fcrepo/rest
106+
```
107+
108+
This variable is necessary for the Crayfish-Commons setup, it has been moved to `/path/to/Homarus/config/packages/crayfish_commons.yaml`
109+
110+
```
111+
crayfish_commons:
112+
fedora_base_uri: 'http://localhost:8080/fcrepo/rest'
113+
```
114+
115+
#### Log settings
116+
Old location `/path/to/Homarus/config/config.yaml`
117+
118+
```
119+
...
120+
log:
121+
# Valid log levels are:
122+
# DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL, ALERT, EMERGENCY, NONE
123+
# log level none won't open logfile
124+
level: DEBUG
125+
file: /var/log/islandora/homarus.log
126+
```
127+
128+
This setting is in `/path/to/Homarus/config/packages/monolog.yaml`. This file contains commented out defaults from Symfony and a new handler for Homarus.
129+
130+
```
131+
monolog:
132+
handlers:
133+
...
134+
homarus:
135+
type: rotating_file
136+
path: /tmp/Homarus.log
137+
level: DEBUG
138+
max_files: 1
139+
channels: ["!event", "!console"]
140+
```
141+
142+
#### Syn settings
143+
Old location `/path/to/Homarus/config/config.yaml`
144+
145+
```
146+
syn:
147+
# toggles JWT security for service
148+
enable: True
149+
# Path to the syn config file for authentication.
150+
# example can be found here:
151+
# https://github.com/Islandora/Syn/blob/main/conf/syn-settings.example$
152+
config: ../syn-settings.xml
153+
```
154+
155+
The `syn.enable` variable is no longer used as Syn is part of the security for Symfony, see [below](#enable-disable-syn) for steps to see where to enable/disable Syn.
156+
157+
The `syn.config` variable is in `/path/to/Homarus/config/crayfish_commons.yaml`.
158+
159+
```
160+
crayfish_commons:
161+
...
162+
#syn_config: '/path/to/syn-settings.xml'
163+
```
164+
165+
`crayfish_commons.syn_config` needs to point to a file or be left commented out to use a default syn config of
166+
167+
```
168+
<?xml version="1.0" encoding="UTF-8"?>
169+
<!-- Default Config -->
170+
<config version='1'>
171+
</config>
172+
```
173+
174+
##### Enable/Disable Syn
175+
To enable/disable Syn look in the [`./config/packages/security.yaml`](config/packages/security.yaml). By default Syn is disabled, to enable look the below lines and follow the included instructions
176+
177+
```
178+
security:
179+
...
180+
firewall:
181+
...
182+
main:
183+
...
184+
# To enable Syn, change anonymous to false and uncomment the lines further below
185+
anonymous: true
186+
...
187+
# To enable Syn, uncomment the below 4 lines and change anonymous to false above.
188+
#provider: jwt_user_provider
189+
#guard:
190+
# authenticators:
191+
# - Islandora\Crayfish\Commons\Syn\JwtAuthenticator
192+
```
193+

0 commit comments

Comments
 (0)