Skip to content

Commit caf0a31

Browse files
vi3itorvi3itor
authored
Native Windows support (#277)
* Compile project on Windows * Fix compilation and run-time issues * Fix runtime errors The problem was with reading files: file paths are different on Unix and Windows. * Set library extension to .pyd * Add platform specific instructions. Build engine using pip Use variable length arrays on platforms other than Windows Build engine using pip on Windows Add more platform-specific instructions * Prepare for PR: add development install, update guides Improve finding Boost Python on Linux. Prepare for PR. Include source files with gfootball_engine Add installation instructions for Windows Add CMakeModules with data_files Use MANIFEST.in to include additional files Remove unnecessary call to Shutdown SDL: it is being called in blunted.cpp on Exit Use platform-specific logic for copying fonts and building Allow development (-e) install on Windows Fix development (-e) install. Update README * * Add GitHub workflow to publish wheels to PyPI * Use vcpkg manifests to install C++ dependencies * Fix tests to run on Windows * Delete Simple Vertex Buffer during Exit (not in the destructor) * Update MANIFEST.in to include necessary files * Add requirements.txt * Update compile engine doc * Install psutil with a separate command in Docker * Update CHANGELOG and docs * Finalize PR changes
1 parent 260cc31 commit caf0a31

45 files changed

Lines changed: 796 additions & 442 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.dockerignore‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1-
third_party/gfootball_engine/build
1+
third_party/gfootball_engine/build*/
22
build/
33
gfootball.egg-info
4+
football-env*
5+
.vs
6+
.idea

‎.github/workflows/publish-wheels.yml‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
name: Build and Publish Python Wheels
2+
3+
on:
4+
release:
5+
types: [published]
6+
7+
jobs:
8+
build-windows:
9+
runs-on: windows-latest
10+
strategy:
11+
fail-fast: false
12+
matrix:
13+
python-version: [ 3.6, 3.7, 3.8, 3.9 ]
14+
architecture: ['x86', 'x64']
15+
16+
env:
17+
VCPKG_ROOT: "${{ github.workspace }}/vcpkg"
18+
19+
steps:
20+
- name: Check out football repository
21+
uses: actions/checkout@v2
22+
with:
23+
path: football
24+
25+
- name: Set up Python
26+
uses: actions/setup-python@v2
27+
with:
28+
python-version: ${{ matrix.python-version }}
29+
architecture: ${{ matrix.architecture }}
30+
31+
- name: Upgrade pip and essential dependencies
32+
run: |
33+
python -m pip install --upgrade pip
34+
pip install -U build setuptools psutil wheel
35+
pip install -r requirements.txt
36+
working-directory: football
37+
38+
- name: Checkout vcpkg
39+
uses: actions/checkout@v2
40+
with:
41+
repository: microsoft/vcpkg
42+
ref: 025e564979cc01d0fbc5c920aa8a36635efb01bb
43+
path: vcpkg
44+
fetch-depth: 0
45+
46+
# Uncomment to speed up building by caching C++ dependencies. Requires `python -m build --wheel` in the next step.
47+
# - name: Restore vcpkg_installed cache
48+
# uses: actions/cache@v2
49+
# with:
50+
# path: ${{ github.workspace }}/football/third_party/gfootball_engine/build_win/vcpkg_installed
51+
# key: vcpkg-${{ runner.os }}-py${{ matrix.python-version }}-${{ matrix.architecture }}-${{ hashFiles('football/third_party/gfootball_engine/vcpkg_manifests/py3*/vcpkg.json') }}
52+
53+
- name: Build package
54+
# `python -m build` without any arguments is used to check if the wheel can be built from source (sdist):
55+
# the tool copies source code to the random temp directory, and proceeds with building.
56+
# Doing so insures that MANIFEST.in contains all the required files to build engine from source.
57+
# The downside is that we can't cache vcpkg_installed packages, since they'll be located in the random temp directory.
58+
# It increases workflow running time from ~15 minutes to ~40 minutes. On the other hand, cache is valid only for 7 days,
59+
# and releases are coming not so often anyway.
60+
# To build in-place and take advantage of vcpkg_installed cache use `python -m build --wheel` instead.
61+
run: python -m build
62+
working-directory: football
63+
64+
- name: Install the package using the wheel
65+
run: python -m pip install temp-gfootball --no-index --find-links=dist
66+
working-directory: football
67+
68+
- name: Run the tests
69+
run: FOR %%f IN (gfootball\env\*test.py) DO ( call python %%f & if errorlevel 1 exit /B 1 )
70+
env:
71+
UNITTEST_IN_DOCKER: 1
72+
shell: cmd
73+
working-directory: football
74+
75+
- uses: actions/upload-artifact@v2
76+
with:
77+
name: windows-wheel-py-${{ matrix.python-version }}-${{ matrix.architecture }}
78+
path: ${{ github.workspace }}/football/dist/*.whl
79+
retention-days: 7
80+
81+
upload-wheels:
82+
needs: [build-windows]
83+
runs-on: ubuntu-latest
84+
steps:
85+
- name: Check out football repository
86+
uses: actions/checkout@v2
87+
88+
- name: Set up Python
89+
uses: actions/setup-python@v2
90+
with:
91+
python-version: '3.x'
92+
93+
- name: Install dependencies
94+
run: python -m pip install --upgrade pip setuptools wheel psutil build
95+
96+
- name: Build sdist
97+
run: python -m build --sdist
98+
99+
- name: Download artifacts
100+
uses: actions/download-artifact@v2
101+
with:
102+
path: ${{ github.workspace }}/artifacts
103+
104+
- name: Move wheels to dist directory
105+
run: mv artifacts/**/*.whl dist
106+
107+
- name: Publish package
108+
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
109+
with:
110+
user: __token__
111+
password: ${{ secrets.PYPI_API_TOKEN }}

‎.gitignore‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,18 @@
1-
third_party/gfootball_engine/build
1+
third_party/gfootball_engine/build*
22
build/
33
gfootball.egg-info
44
__pycache__
55
*.a
66
*.so
77
*.dylib
8-
CmakeCache.txt
8+
C[M,m]akeCache.txt
99
third_party/gfootball_engine/CMakeFiles/
1010
third_party/gfootball_engine/Makefile
1111
third_party/gfootball_engine/cmake_install.cmake
1212
third_party/gfootball_engine/gameplayfootball
1313
footballenv/
14-
football-env/
14+
football-env*/
1515
cmake-build-debug/
1616
.idea
1717
.vs
18+
vcpkg_installed/

‎CHANGELOG‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,10 @@ should not change, as modifications made to the environment are either
55
new features or backward compatible bug fixes. We will maintain vX branches
66
pointing at the most recent vX.Y.
77

8+
v2.10
9+
- Windows support.
10+
- Development mode (pip install -e .)
11+
812
v2.9
913
- Drop hard dependency on Tensorflow.
1014
- Make installation easier, support for Ubuntu 20.04.

‎Dockerfile‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,10 @@ RUN apt-get update && apt-get --no-install-recommends install -yq git cmake buil
66
libgl1-mesa-dev libsdl2-dev \
77
libsdl2-image-dev libsdl2-ttf-dev libsdl2-gfx-dev libboost-all-dev \
88
libdirectfb-dev libst-dev mesa-utils xvfb x11vnc \
9-
libsdl-sge-dev python3-pip
9+
python3-pip
1010

11-
RUN python3 -m pip install --upgrade pip setuptools psutil
11+
RUN python3 -m pip install --upgrade pip setuptools wheel
12+
RUN python3 -m pip install psutil
1213

1314
COPY . /gfootball
1415
RUN cd /gfootball && python3 -m pip install .

‎Dockerfile_examples‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,9 @@ RUN apt-get update && apt-get --no-install-recommends install -yq git cmake buil
77
libgl1-mesa-dev libsdl2-dev \
88
libsdl2-image-dev libsdl2-ttf-dev libsdl2-gfx-dev libboost-all-dev \
99
libdirectfb-dev libst-dev mesa-utils xvfb x11vnc \
10-
libsdl-sge-dev python3-pip
10+
python3-pip
1111

12-
RUN python3 -m pip install --upgrade pip setuptools
12+
RUN python3 -m pip install --upgrade pip setuptools wheel
1313
RUN pip3 install --no-cache-dir psutil dm-sonnet==1.*
1414
RUN pip list | grep 'tensorflow ' || pip3 install --no-cache-dir tensorflow==1.15.*
1515

‎MANIFEST.in‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,16 @@
1-
include gfootball/build_game_engine.sh
2-
recursive-include third_party/gfootball_engine/ *
3-
recursive-include third_party/fonts/ *
1+
include gfootball/build_game_engine.*
2+
recursive-include gfootball/colabs *
3+
recursive-include gfootball/doc *
4+
recursive-include gfootball/eval_server/proto *
5+
recursive-include gfootball/examples *
6+
7+
include Dockerfile .dockerignore CHANGELOG CONTRIBUTING.md run_docker_test.sh
8+
9+
recursive-include third_party/fonts *
10+
recursive-include third_party/gfootball_engine *
11+
recursive-exclude third_party/gfootball_engine *.a libgame.so cmake_install.cmake Makefile
12+
prune third_party/gfootball_engine/CMakeFiles
13+
prune third_party/gfootball_engine/cmake-build-debug
14+
prune third_party/gfootball_engine/build*
15+
prune third_party/gfootball_engine/.vs
16+
prune third_party/gfootball_engine/.idea

‎README.md‎

Lines changed: 30 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -36,50 +36,63 @@ Instructions are available [here](gfootball/doc/docker.md).
3636

3737
#### 1. Install required packages
3838
#### Linux
39-
```
39+
```shell
4040
sudo apt-get install git cmake build-essential libgl1-mesa-dev libsdl2-dev \
4141
libsdl2-image-dev libsdl2-ttf-dev libsdl2-gfx-dev libboost-all-dev \
42-
libdirectfb-dev libst-dev mesa-utils xvfb x11vnc libsdl-sge-dev python3-pip
42+
libdirectfb-dev libst-dev mesa-utils xvfb x11vnc python3-pip
4343

44-
python3 -m pip install --upgrade pip setuptools psutil
44+
python3 -m pip install --upgrade pip setuptools psutil wheel
4545
```
4646

47-
#### Mac OS X
47+
#### macOS
4848
First install [brew](https://brew.sh/). It should automatically install Command Line Tools.
4949
Next install required packages:
5050

51-
```
51+
```shell
5252
brew install git python3 cmake sdl2 sdl2_image sdl2_ttf sdl2_gfx boost boost-python3
53+
54+
python3 -m pip install --upgrade pip setuptools psutil wheel
5355
```
5456

55-
#### 2a. From PyPi package
57+
#### Windows
58+
Install [Git](https://git-scm.com/download/win) and [Python 3](https://www.python.org/downloads/).
59+
Update pip in the Command Line (here and for the **next steps** type `python` instead of `python3`)
60+
```commandline
61+
python -m pip install --upgrade pip setuptools psutil wheel
5662
```
63+
64+
#### 2. Install GFootball
65+
#### 2a. From PyPi package
66+
```shell
5767
python3 -m pip install gfootball
5868
```
5969

6070
#### 2b. Installing from sources using GitHub repository
6171

62-
```
72+
```shell
6373
git clone https://github.com/google-research/football.git
6474
cd football
6575
```
6676

6777
Optionally you can use [virtual environment](https://docs.python.org/3/tutorial/venv.html):
6878

69-
```
79+
```shell
7080
python3 -m venv football-env
7181
source football-env/bin/activate
7282
```
7383

74-
The last step is to build the environment:
84+
Next, build the game engine and install dependencies:
7585

76-
```
86+
```shell
7787
python3 -m pip install .
7888
```
7989
This command can run for a couple of minutes, as it compiles the C++ environment in the background.
8090

91+
To compile the engine on Windows you have to install additional tools and set environment variables. See
92+
[Compiling Engine](gfootball/doc/compile_engine.md#windows) for detailed instructions.
93+
8194
#### 3. Time to play!
82-
```
95+
```shell
8396
python3 -m gfootball.play_game --action_set=full
8497
```
8598
Make sure to check out the [keyboard mappings](#keyboard-mappings).
@@ -106,11 +119,11 @@ To quit the game press Ctrl+C in the terminal.
106119
### Run training
107120
In order to run TF training, you need to install additional dependencies
108121

109-
- Update PIP, so that tensorflow 1.15 is available: `python3 -m pip install --upgrade pip setuptools`
122+
- Update PIP, so that tensorflow 1.15 is available: `python3 -m pip install --upgrade pip setuptools wheel`
110123
- TensorFlow: `python3 -m pip install tensorflow==1.15.*` or
111124
`python3 -m pip install tensorflow-gpu==1.15.*`, depending on whether you want CPU or
112125
GPU version;
113-
- Sonnet: `python3 -m pip install dm-sonnet==1.*`;
126+
- Sonnet and psutil: `python3 -m pip install dm-sonnet==1.* psutil`;
114127
- OpenAI Baselines:
115128
`python3 -m pip install git+https://github.com/openai/baselines.git@master`.
116129

@@ -131,7 +144,8 @@ In order to reproduce PPO results from the paper, please refer to:
131144

132145
## Playing the game
133146

134-
Please note that playing the game is implemented through an environment, so human-controlled players use the same interface as the agents. One important implication is that there is a single action per 100 ms reported to the environment, which might cause a lag effect when playing.
147+
Please note that playing the game is implemented through an environment, so human-controlled players use the same interface as the agents.
148+
One important implication is that there is a single action per 100 ms reported to the environment, which might cause a lag effect when playing.
135149

136150

137151
### Keyboard mappings
@@ -143,8 +157,8 @@ The game defines following keyboard mapping (for the `keyboard` player type):
143157
* `ARROW RIGHT` - run to the right.
144158
* `S` - short pass in the attack mode, pressure in the defense mode.
145159
* `A` - high pass in the attack mode, sliding in the defense mode.
146-
* `D` - shot in the the attack mode, team pressure in the defense mode.
147-
* `W` - long pass in the the attack mode, goalkeeper pressure in the defense mode.
160+
* `D` - shot in the attack mode, team pressure in the defense mode.
161+
* `W` - long pass in the attack mode, goalkeeper pressure in the defense mode.
148162
* `Q` - switch the active player in the defense mode.
149163
* `C` - dribble in the attack mode.
150164
* `E` - sprint.

‎gfootball/build_game_engine.bat‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
@echo off
2+
rem This batch file compiles gfootball engine on Windows
3+
4+
if "%VCPKG_ROOT%"=="" (
5+
echo VCPKG_ROOT environment variable is missing
6+
echo Execute: SET "VCPKG_ROOT=PATH_TO_VCPKG" e.g., SET "VCPKG_ROOT=C:\src\vcpkg\"
7+
exit /b 1
8+
)
9+
if "%GENERATOR_PLATFORM%"=="" (
10+
echo GENERATOR_PLATFORM environment variable is missing
11+
echo Execute: SET GENERATOR_PLATFORM={x64 or Win32}, depending on the Python Interpreter
12+
exit /b 1
13+
)
14+
if "%BUILD_CONFIGURATION%"=="" set BUILD_CONFIGURATION=Release
15+
16+
pushd third_party\gfootball_engine
17+
18+
if not exist build_win mkdir build_win
19+
if exist build_win\CMakeCache.txt del build_win\CMakeCache.txt
20+
if exist build_win\%BUILD_CONFIGURATION% rmdir /s /q build_win\%BUILD_CONFIGURATION%
21+
22+
pushd build_win
23+
cmake .. -A%GENERATOR_PLATFORM%
24+
if errorlevel 1 exit /B 1
25+
26+
cmake --build . --parallel --config %BUILD_CONFIGURATION%
27+
if errorlevel 1 exit /B 1
28+
29+
popd && popd

0 commit comments

Comments
 (0)