Skip to content

Commit 54afe50

Browse files
cyrushcyrush
authored
misc changes (#1543)
* add notes * docker multi arch fix * update docker build recipe
1 parent 114c16e commit 54afe50

8 files changed

Lines changed: 104 additions & 23 deletions

File tree

‎AGENTS.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
HUMANS.md

‎HUMANS.md‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Conduit repo notes
2+
3+
## Repo layout
4+
- Source lives under `src/` (top-level `CMakeLists.txt` expects `-S src`).
5+
- Libraries are implemented in `src/libs`.
6+
- Tests are implemented in `src/tests`.
7+
- Documentation is written using sphinx and is located at `src/docs/sphinx`.
8+
- Conduit is developed on github and the repo location is https://github.com/llnl/conduit.
9+
10+
11+
## Third-party Dependencies
12+
- You can use `scripts/build_conduit/build_conduit.sh` to build the third party libraries that Conduit requires
13+
14+
## Build (CMake)
15+
- Prefer out-of-source builds:
16+
- Configure: `cmake -S src -B build-debug -C host-config.cmake -DCMAKE_BUILD_TYPE=Debug`
17+
- Build: `cmake --build build-debug -j`
18+
- Test (ctest): `ctest --test-dir build-debug --output-on-failure`
19+
- You can run Conduit's python module from a build using `bin/run_python_with_conduit_build.sh`
20+
21+
22+
## Tests
23+
- To run a single test: `ctest --test-dir build-debug -R '^t_conduit_node$' --output-on-failure -V`
24+
- Keep new tests small and deterministic; add them to the closest existing `src/tests/**/CMakeLists.txt`.
25+
26+
27+
## Style / scope
28+
- Prefer minimal, surgical changes; don’t reformat unrelated code.
29+
- Avoid adding new dependencies or touching vendored third-party code unless required by the task.
30+
31+
32+
## Conduit
33+
- The conduit library/module provides the core Conduit API.
34+
- Conduit provides C++, Python, C, and Fortran APIs.
35+
- The main data structure in Conduit API is Node object, it is tree based data structure similar to a json object.
36+
- Leaf nodes hold 1D arrays described using bitwidth-style types or c-style strings.
37+
- Leaf data can be copied into the tree using set or the assignment operator.
38+
- Leaf data can be described zero-copy using set_external.
39+
- Conduit Nodes can be converted to and loaded from yaml or json strings using `to_yaml`, `to_json`, and `parse`.
40+
41+
42+
## Blueprint
43+
- The blueprint library/module outlines schemas and convention for sharing data using Conduit Nodes.
44+
- Conduit provides conventions for sharing simulation mesh data that are called the Mesh Blueprint.
45+
- You can use `conduit.blueprint.mesh.verify` to check if a Conduit Node confirms to these conventions.
46+
- You can generate many example meshes using `conduit.blueprint.mesh.examples.generate`, you can review the mesh types and their options using `generate_default_options`.
47+
- You can save meshes to hdf5 or yaml using `conduit.relay.io.blueprint.save_mesh`.
48+
- You can load meshes from hdf5 or yaml using `conduit.relay.io.blueprint.load_mesh`.
49+
50+
51+
## Relay
52+
- The relay library/module provides I/O methods for using Conduit Nodes with hdf5, silo, and mpi.

‎azure-pipelines.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -522,7 +522,7 @@ stages:
522522
####################
523523
# run docker build
524524
####################
525-
cd src/examples/docker/ubuntu/ && ./example_build.sh
525+
cd src/examples/docker/ubuntu/ && ./build.sh
526526
displayName: 'Docker Build'
527527
528528

‎src/docs/sphinx/building.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -403,7 +403,7 @@ Under ``src/examples`` there are examples demonstrating how to use Conduit in a
403403
Building Conduit in a Docker Container
404404
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
405405

406-
Under ``src/examples/docker/ubuntu`` there is an example ``Dockerfile`` which can be used to create an ubuntu-based docker image with a build of the Conduit. There is also a script that demonstrates how to build a Docker image from the Dockerfile (``example_build.sh``) and a script that runs this image in a Docker container (``example_run.sh``). The Conduit repo is cloned into the image's file system at ``/conduit``, the build directory is ``/conduit/build-debug``, and the install directory is ``/conduit/install-debug``.
406+
Under ``src/examples/docker/ubuntu`` there is an example ``Dockerfile`` which can be used to create an ubuntu-based docker image with a build of the Conduit. There is also a script that demonstrates how to build a Docker image from the Dockerfile (``build.sh``) and a script that runs this image in a Docker container (``run.sh``). The Conduit repo is cloned into the image's file system at ``/conduit``, the build directory is ``/conduit/build``, and the install directory is ``/conduit/install``.
407407

408408
.. _building_with_pip:
409409

‎src/examples/docker/ubuntu/Dockerfile‎

Lines changed: 40 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
FROM ubuntu:24.04
66

7+
ARG TARGETARCH
8+
79
#####################
810
# base deps
911
#####################
@@ -57,52 +59,72 @@ RUN apt-get update && apt-get -y install \
5759

5860
# manual mpich install
5961
# https://github.com/actions/runner-images/issues/13204
60-
RUN curl -LO https://launchpad.net/ubuntu/+source/mpich/4.2.0-5.1/+build/28285882/+files/libmpich12_4.2.0-5.1_amd64.deb
61-
RUN curl -LO https://launchpad.net/ubuntu/+source/mpich/4.2.0-5.1/+build/28285882/+files/mpich_4.2.0-5.1_amd64.deb
62-
RUN curl -LO https://launchpad.net/ubuntu/+source/mpich/4.2.0-5.1/+build/28285882/+files/libmpich-dev_4.2.0-5.1_amd64.deb
62+
#
63+
# amd64: curl -LO https://launchpad.net/ubuntu/+source/mpich/4.2.0-5.1/+build/28285882/
64+
# arm64: curl -LO https://launchpad.net/ubuntu/+source/mpich/4.2.0-5.1/+build/28285883/
65+
66+
RUN case "$TARGETARCH" in \
67+
"amd64") BUILD_NUM="28285882" ;; \
68+
"arm64") BUILD_NUM="28285883" ;; \
69+
esac; \
70+
curl -LO https://launchpad.net/ubuntu/+source/mpich/4.2.0-5.1/+build/${BUILD_NUM}/+files/libmpich12_4.2.0-5.1_${TARGETARCH}.deb
71+
72+
RUN case "$TARGETARCH" in \
73+
"amd64") BUILD_NUM="28285882" ;; \
74+
"arm64") BUILD_NUM="28285883" ;; \
75+
esac; \
76+
curl -LO https://launchpad.net/ubuntu/+source/mpich/4.2.0-5.1/+build/${BUILD_NUM}/+files/mpich_4.2.0-5.1_${TARGETARCH}.deb
77+
78+
RUN case "$TARGETARCH" in \
79+
"amd64") BUILD_NUM="28285882" ;; \
80+
"arm64") BUILD_NUM="28285883" ;; \
81+
esac; \
82+
curl -LO https://launchpad.net/ubuntu/+source/mpich/4.2.0-5.1/+build/${BUILD_NUM}/+files/libmpich-dev_4.2.0-5.1_${TARGETARCH}.deb
83+
6384

64-
RUN apt install -y ./libmpich12_4.2.0-5.1_amd64.deb
65-
RUN apt install -y ./mpich_4.2.0-5.1_amd64.deb
66-
RUN apt install -y ./libmpich-dev_4.2.0-5.1_amd64.deb
85+
RUN apt install -y ./libmpich12_4.2.0-5.1_${TARGETARCH}.deb
86+
RUN apt install -y ./mpich_4.2.0-5.1_${TARGETARCH}.deb
87+
RUN apt install -y ./libmpich-dev_4.2.0-5.1_${TARGETARCH}.deb
6788

6889
RUN rm -rf /var/lib/apt/lists/*
6990

70-
# untar the current source (created as part of example_build.sh)
91+
# untar the current source (created as part of build.sh)
7192
COPY conduit.docker.src.tar /
7293
RUN tar -xf conduit.docker.src.tar
7394

7495
# if you would like to clone conduit directly, you can use:
7596
#RUN git clone --depth 1 https://github.com/LLNL/conduit.git
7697

77-
# at some sites ssl certs are intercepted, which cases issues fetching
98+
# at some sites ssl certs are intercepted, which cases issues fetching
7899
# tpl sources via https
79100

80101
# to resolve this, either:
81-
# 1) pass the "-k" option to uberenv (recommended),
102+
# 1) pass the "-k" option to uberenv (recommended),
82103
# 2) install the proper certs into the image, or
83-
# 3) use the following commands to disable ssl for git and
104+
# 3) use the following commands to disable ssl for git and
84105
# curl (both are used by spack):
85106
#RUN git config --global http.sslVerify false
86107
#RUN echo insecure >> ~/.curlrc
87108

88109
# bootstrap third party libs using spack and uberenv
89110
RUN cd conduit && python3 scripts/uberenv/uberenv.py \
90111
-k \
112+
--prefix=/uberenv_libs \
91113
--spec "+mpi%gcc" \
92114
--spack-env-file=scripts/uberenv_configs/spack_configs/envs/ubuntu-24.04/spack.yaml
93115

94116
# configure a debug build with cmake
95117
RUN cd conduit && \
96-
uberenv_libs/spack/opt/spack/*/cmake-*/bin/cmake \
97-
-DCMAKE_BUILD_TYPE=Debug \
98-
-DCMAKE_INSTALL_PREFIX=/conduit/install-debug \
99-
-C /conduit/uberenv_libs/*-patch.cmake \
100-
-B build-debug -S src
118+
/uberenv_libs/spack/opt/spack/*/cmake-*/bin/cmake \
119+
-DCMAKE_BUILD_TYPE=Release \
120+
-DCMAKE_INSTALL_PREFIX=/conduit/install \
121+
-C /uberenv_libs/*-patch.cmake \
122+
-B build -S src
101123

102124
# build, test, and install conduit
103-
RUN cd conduit/build-debug && make
104-
RUN cd conduit/build-debug && env make test
105-
RUN cd conduit/build-debug && make install
125+
RUN cmake --build conduit/build --config Release
126+
RUN cmake -E env CTEST_OUTPUT_ON_FAILURE=1 cmake --build conduit/build
127+
RUN cmake --install conduit/build --config Release
106128

107129
# open port 9000, for use by conduit relay
108130
EXPOSE 9000

src/examples/docker/ubuntu/example_build.sh renamed to src/examples/docker/ubuntu/build.sh

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22
# Project developers. See top-level LICENSE AND COPYRIGHT files for dates and
33
# other details. No copyright assignment is required to contribute to Conduit.
44

5+
set -ev
6+
57
# remove old source tarball if it exists
68
echo "rm -f conduit.docker.src.tar.gz"
79
rm -f conduit.docker.src.tar.gz
@@ -14,7 +16,11 @@ cd ../../../../ && python3 package.py src/examples/docker/ubuntu/conduit.docker.
1416
echo "cd src/examples/docker/ubuntu/"
1517
cd src/examples/docker/ubuntu/
1618

19+
export TAG_ARCH=`uname -m`
20+
export TAG_NAME=conduit:ubuntu-24.04-${TAG_ARCH}
1721

1822
# exec docker build to create image
19-
echo "docker build -t conduit-ubuntu:current ."
20-
docker build -t conduit-ubuntu:current .
23+
echo "docker build -t ${TAG_NAME} ."
24+
docker build -t ${TAG_NAME} .
25+
26+

src/examples/docker/ubuntu/example_run.sh renamed to src/examples/docker/ubuntu/run.sh

File renamed without changes.

‎src/libs/conduit/python/run_python_with_conduit.sh.in‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
#!/bin/bash
2-
env PYTHONPATH=$PYTHONPATH:@_PYMOD_DIR@ @PYTHON_EXECUTABLE@ $@
2+
env PYTHONPATH=$PYTHONPATH:@_PYMOD_DIR@ @PYTHON_EXECUTABLE@ "$@"

0 commit comments

Comments
 (0)