Skip to content

Commit 1424fec

Browse files
jsturtevantjsturtevant
authored
Merge pull request #4 from jsturtevant/update-witbindgen
Update Wit-bindgen, related tools and make tests pass
2 parents b3a97cb + d40e0bc commit 1424fec

File tree

26 files changed

+219
-144
lines changed

26 files changed

+219
-144
lines changed

‎.github/workflows/build.yml

+1-3
Original file line numberDiff line numberDiff line change
@@ -16,16 +16,14 @@ jobs:
1616

1717
steps:
1818
- uses: actions/checkout@v3
19-
with:
20-
submodules: 'true'
2119
- name: Setup .NET
2220
uses: actions/setup-dotnet@v3
2321
with:
2422
dotnet-version: 8.0.100-rtm.23516.5
2523
- name: Restore dependencies
2624
run: dotnet restore
2725
- name: Prepare WASM SDKs
28-
run: dotnet msbuild src/WasmComponent.Sdk/build/WasmComponent.Sdk.targets /t:PrepareWasmSdks
26+
run: dotnet msbuild src/WitBindgen/build/WitBindgen.targets /t:PrepareWasmSdks
2927
- name: Build
3028
run: dotnet build --no-restore /p:BuildNumber=${{ github.run_number }}
3129
- name: Test

‎.gitmodules

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[submodule "modules/wit-bindgen"]
22
path = modules/wit-bindgen
3-
url = https://github.com/jsturtevant/wit-bindgen.git
3+
url = https://github.com/bytecodealliance/wit-bindgen.git
44
[submodule "modules/wasm-tools"]
55
path = modules/wasm-tools
6-
url = https://github.com/dicej/wasm-tools.git
6+
url = https://github.com/bytecodealliance/wasm-tools.git

‎CONTRIBUTING.md

+58-6
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,16 @@ Open an issue detailing the issue you've encountered or feature you would like t
33

44
Bug fixes and new features must be submitted using a pull request and pass CI to be included in the project.
55

6-
## Building the project locally
6+
## Building
77

88
Requires [.NET 8+](https://dotnet.microsoft.com/en-us/download)
99

1010
```
11-
dotnet msbuild src/WasmComponent.Sdk/build/WasmComponent.Sdk.targets /t:PrepareWasmSdks
12-
git submodule update --init
11+
## needed to avoid errors with multiple projects calling and downloading the sdks at same time (https://github.com/bytecodealliance/componentize-dotnet/issues/8)
12+
dotnet msbuild src/WitBindgen/build/WitBindgen.targets /t:PrepareWasmSdks
1313
dotnet build
1414
```
1515

16-
If you are experiencing issues with values not being updated, try running `dotnet clean` and using the steps above
17-
1816
## Testing
1917

2018
Run the tests:
@@ -23,5 +21,59 @@ Run the tests:
2321
dotnet test
2422
```
2523

24+
> tip: If you've already built the project you can speed up the tests by running `dotnet test --no-build`
25+
26+
## Build Wasm tools locally
27+
28+
> requires [rust](https://www.rust-lang.org/tools/install)
29+
30+
The project is configured by default to pull tools such as [wit-bindgen](https://github.com/bytecodealliance/wit-bindgen) from their releases. It is possible to use custom builds of these tools via submodules:
31+
32+
```
33+
## get submodules
34+
git submodule update --int
35+
36+
## get latest code from configured branch
37+
git submodule update --recursive --remote
38+
39+
## optional, change the branch for the project
40+
cd modules/<project>
41+
git checkout <branch>
42+
```
43+
44+
Modify the [WasmComponentSdk.csproj](./src/WasmComponent.Sdk/WasmComponent.Sdk.csproj) to enable building from source:
45+
46+
```
47+
<BuildWasmToolsLocally>true</BuildWasmToolsLocally>
48+
```
49+
50+
Modify the [WitBindgen.csproj](./src/WitBindgen/WitBindgen.csproj) to enable building from source:
51+
52+
```
53+
<BuildWitBindgenLocally>true</BuildWitBindgenLocally>
54+
```
55+
56+
And then follow the [project build steps](#building).
57+
58+
### Debugging
59+
60+
Create a msbuild debug log:
61+
62+
```
63+
dotnet build /bl
64+
```
65+
66+
View the log with https://www.msbuildlog.com/.
67+
68+
Learn more at [trouble shooting techniques](https://learn.microsoft.com/en-us/visualstudio/ide/msbuild-logs?view=vs-2022) for msbuild.
69+
2670
## Getting help
27-
While we work on improving the documentation for contributing, if you have any questions please drop a note in the [c# zulip chat](https://bytecodealliance.zulipchat.com/#narrow/stream/407028-C.23.2F.2Enet-collaboration).
71+
72+
> If you have any questions please drop a note in the [c# zulip chat](https://bytecodealliance.zulipchat.com/#narrow/stream/407028-C.23.2F.2Enet-collaboration).
73+
74+
This project uses MSbuild and .NET Project SDKS. Learn more about this tooling in:
75+
76+
- [MSbuild docs](https://learn.microsoft.com/en-us/visualstudio/msbuild/msbuild?view=vs-2022)
77+
- [.NET Project SDKS](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/overview)
78+
- [Creating Reliable Builds](https://learn.microsoft.com/en-us/archive/msdn-magazine/2009/february/msbuild-best-practices-for-creating-reliable-builds-part-1#id0090093)
79+

‎modules/wasm-tools

Submodule wasm-tools updated 4709 files

‎modules/wit-bindgen

Submodule wit-bindgen updated 221 files

‎samples/calculator/Adder/OperationsImpl.cs

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
namespace wit_computer.Wit.exports.example.calculator.Operations;
1+
namespace ComputerWorld.wit.exports.example.calculator;
22

3-
public class OperationsImpl : Operations
3+
public class OperationsImpl : IOperations
44
{
55
public static int Add(int left, int right)
66
{

‎samples/calculator/CalculatorComposed/CalculatorComposed.csproj

+2-2
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,8 @@
1717

1818
<Target Name="ComposeWasmComponent" AfterTargets="Build">
1919
<PropertyGroup>
20-
<EntrypointComponent>../CalculatorHost/bin/$(Configuration)/$(TargetFramework)/wasi-wasm/native/CalculatorHost.component.wasm</EntrypointComponent>
21-
<DependencyComponent>../Adder/bin/$(Configuration)/$(TargetFramework)/wasi-wasm/native/Adder.component.wasm</DependencyComponent>
20+
<EntrypointComponent>../CalculatorHost/bin/$(Configuration)/$(TargetFramework)/wasi-wasm/native/calculatorhost-component.wasm</EntrypointComponent>
21+
<DependencyComponent>../Adder/bin/$(Configuration)/$(TargetFramework)/wasi-wasm/native/adder-component.wasm</DependencyComponent>
2222
</PropertyGroup>
2323

2424
<MakeDir Directories="dist" />

‎samples/calculator/CalculatorHost/Program.cs

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
using wit_hostapp.Wit.imports.example.calculator.Operations;
1+
using HostappWorld.wit.imports.example.calculator;
22

33
var left = 123;
44
var right = 456;

‎src/WasmComponent.Sdk/WasmComponent.Sdk.csproj

+6-9
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,10 @@
99

1010
<!-- Things you might want to edit -->
1111
<!-- Set BuildWasmToolsLocally to true if you want to build modules/wasm-tools locally and use its output -->
12-
<BuildWasmToolsLocally>true</BuildWasmToolsLocally>
13-
<PrebuiltWasmToolsVersion>1.0.51</PrebuiltWasmToolsVersion>
14-
<PrebuiltWasmToolsBaseUrl>https://github.com/bytecodealliance/wasm-tools/releases/download/wasm-tools-$(PrebuiltWasmToolsVersion)/wasm-tools-$(PrebuiltWasmToolsVersion)</PrebuiltWasmToolsBaseUrl>
15-
<WasmtimeVersionForWasiSnapshotPreview1Adapters>14.0.4</WasmtimeVersionForWasiSnapshotPreview1Adapters>
12+
<BuildWasmToolsLocally>false</BuildWasmToolsLocally>
13+
<PrebuiltWasmToolsVersion>1.209.1</PrebuiltWasmToolsVersion>
14+
<PrebuiltWasmToolsBaseUrl>https://github.com/bytecodealliance/wasm-tools/releases/download/v$(PrebuiltWasmToolsVersion)/wasm-tools-$(PrebuiltWasmToolsVersion)</PrebuiltWasmToolsBaseUrl>
15+
<WasmtimeVersionForWasiSnapshotPreview1Adapters>21.0.1</WasmtimeVersionForWasiSnapshotPreview1Adapters>
1616

1717
<WasmToolsModuleRoot>$(MSBuildThisFileDirectory)..\..\modules\wasm-tools\</WasmToolsModuleRoot>
1818

@@ -68,13 +68,10 @@
6868
<RemoveDir Directories="tools\temp" />
6969
</Target>
7070

71-
<Target Name="DownloadWasiPreview1Adapters" Condition="!Exists('tools\wasi-wasm\lastbuild.txt')">
71+
<Target Name="DownloadWasiPreview1Adapters" Inputs="$(MSBuildThisFileDirectory)tools\wasi-wasm\version" Outputs="$(MSBuildThisFileDirectory)tools\wasi-wasm\version" >
7272
<DownloadFile SourceUrl="https://github.com/bytecodealliance/wasmtime/releases/download/v$(WasmtimeVersionForWasiSnapshotPreview1Adapters)/wasi_snapshot_preview1.command.wasm" DestinationFolder="tools\wasi-wasm" />
7373
<DownloadFile SourceUrl="https://github.com/bytecodealliance/wasmtime/releases/download/v$(WasmtimeVersionForWasiSnapshotPreview1Adapters)/wasi_snapshot_preview1.reactor.wasm" DestinationFolder="tools\wasi-wasm" />
74-
<WriteLinesToFile File="tools\wasi-wasm\lastbuild.txt" Lines="" Overwrite="true" />
75-
<ItemGroup>
76-
<FileWrites Include="tools\wasi-wasm\lastbuild.txt" />
77-
</ItemGroup>
74+
<WriteLinesToFile File="$(MSBuildThisFileDirectory)tools\wasi-wasm\version" Lines="$(WasmtimeVersionForWasiSnapshotPreview1Adapters)" Overwrite="true" WriteOnlyWhenDifferent="true" />
7875
</Target>
7976

8077
<Target Name="PackTaskDependencies" BeforeTargets="GenerateNuspec">

‎src/WasmComponent.Sdk/build/WasmComponent.Sdk.targets

+5-80
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,4 @@
11
<Project>
2-
<!--
3-
MSBuild stuff to acquire the necessary SDKs (WASI SDK and Emscripten) automatically. It will take a few mins on the
4-
first build on a given machine, but after that should no-op.
5-
-->
6-
7-
<PropertyGroup>
8-
<!-- Keep this block all in sync manually, since URLs can be arbitrary -->
9-
<WasiSdkVersion>20.0</WasiSdkVersion>
10-
<WasiSdkUrl Condition="$([MSBuild]::IsOSPlatform('Windows'))">https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0.m-mingw.tar.gz</WasiSdkUrl>
11-
<WasiSdkUrl Condition="$([MSBuild]::IsOSPlatform('Linux'))">https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0-linux.tar.gz</WasiSdkUrl>
12-
<WasiSdkUrl Condition="$([MSBuild]::IsOSPlatform('OSX'))">https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0-macos.tar.gz</WasiSdkUrl>
13-
<WasiSdkRoot>$([System.IO.Path]::Combine("$([System.Environment]::GetFolderPath(SpecialFolder.UserProfile))", ".wasi-sdk", "wasi-sdk-$(WasiSdkVersion)"))</WasiSdkRoot>
14-
15-
<EmSdkVersion>3.1.23</EmSdkVersion>
16-
<EmSdkUrl>https://github.com/emscripten-core/emsdk/archive/refs/tags/$(EmSdkVersion).zip</EmSdkUrl>
17-
<!-- Support bring your own emscripten if $(EMSDK) is already set-->
18-
<EmscriptenRoot Condition="'$(EMSDK)' == ''">$([System.IO.Path]::Combine("$([System.Environment]::GetFolderPath(SpecialFolder.UserProfile))", ".emsdk", "emsdk-$(EmSdkVersion)"))</EmscriptenRoot>
19-
<EmscriptenRoot Condition="'$(EMSDK)' != ''">$(EMSDK)</EmscriptenRoot>
20-
</PropertyGroup>
21-
222
<Target Name="EmitWasmOnBuild" AfterTargets="CopyFilesToOutputDirectory" DependsOnTargets="LinkNativeLlvm; ConvertToWasmComponent"
233
Condition="'$(RuntimeIdentifier)' == 'wasi-wasm'">
244
<Message Importance="high" Text="$(ProjectName) -> $([System.IO.Path]::GetFullPath('$(NativeComponentBinary)'))" />
@@ -33,66 +13,11 @@
3313
<WasiPreview1AdapterType>reactor</WasiPreview1AdapterType>
3414
<WasiPreview1AdapterType Condition="'$(OutputType.ToLower())' == 'exe'">command</WasiPreview1AdapterType>
3515
<WasiPreview1AdapterPath>$(MSBuildThisFileDirectory)../tools/wasi-wasm/wasi_snapshot_preview1.$(WasiPreview1AdapterType).wasm</WasiPreview1AdapterPath>
36-
<NativeComponentBinary>$(NativeOutputPath)$(TargetName).component.wasm</NativeComponentBinary>
37-
</PropertyGroup>
38-
</Target>
39-
40-
<Target Name="PrepareWasmSdks" BeforeTargets="CheckWasmSdks" DependsOnTargets="ObtainWasiSdk; ObtainEmscripten">
41-
<PropertyGroup>
42-
<EmSdk>$(EmscriptenRoot)</EmSdk>
43-
<WASI_SDK_PATH>$(WasiSdkRoot)</WASI_SDK_PATH>
44-
</PropertyGroup>
45-
</Target>
46-
47-
<Target Name="ObtainEmscripten" Condition="'$(EMSDK)' == '' AND !(Exists($(EmscriptenRoot)))">
48-
<!--
49-
This is not ideal because if your solution has multiple projects that use WasmComponent.Sdk, then if you
50-
build in parallel in CI where your machine doesn't already have wasi-sdk/emsdk, then it may try to download
51-
and extract the SDKs multiple times in parallel to the same disk location, which may cause it to fail.
52-
The only reason this doesn't happen in this repo is that it explicitly runs the PrepareWasmSdks task before
53-
building other projects.
54-
55-
For a proper fix, consider implementing an MSBuild task in C# that obtains wasi-sdk/emsdk, and uses a mutex
56-
so that only one flow executes at a time, with others blocking until it's done.
57-
-->
58-
59-
<PropertyGroup>
60-
<EmSdkDownloadTempDir>$([System.IO.Path]::Combine($([System.IO.Path]::GetTempPath()), $([System.IO.Path]::GetRandomFileName())))</EmSdkDownloadTempDir>
16+
<!--
17+
wasm compose requires kabab case (todo: revisit when move to wac https://github.com/bytecodealliance/componentize-dotnet/issues/5)
18+
https://github.com/bytecodealliance/wasm-tools/issues/1440
19+
-->
20+
<NativeComponentBinary>$(NativeOutputPath)$(TargetName.ToLower())-component.wasm</NativeComponentBinary>
6121
</PropertyGroup>
62-
63-
<MakeDir Directories="$(EmSdkDownloadTempDir)" />
64-
<DownloadFile
65-
SourceUrl="$(EmSdkUrl)"
66-
DestinationFolder="$(EmSdkDownloadTempDir)">
67-
<Output TaskParameter="DownloadedFile" ItemName="EmSdkDownloadTempFile" />
68-
</DownloadFile>
69-
70-
<!-- Windows 10+ has tar built in, so this should work cross-platform -->
71-
<Message Importance="high" Text="Extracting @(EmSdkDownloadTempFile) to $(EmscriptenRoot)..." />
72-
<MakeDir Directories="$(EmscriptenRoot)" />
73-
<Exec Command="tar -xf &quot;@(EmSdkDownloadTempFile)&quot; -C . --strip-components=1" WorkingDirectory="$(EmscriptenRoot)" />
74-
<RemoveDir Directories="$(EmSdkDownloadTempDir)" />
75-
76-
<Exec Command="emsdk install $(EmSdkVersion)" WorkingDirectory="$(EmscriptenRoot)" />
77-
<Exec Command="emsdk activate $(EmSdkVersion)" WorkingDirectory="$(EmscriptenRoot)" />
78-
</Target>
79-
80-
<Target Name="ObtainWasiSdk" Condition="!(Exists($(WasiSdkRoot)))">
81-
<PropertyGroup>
82-
<WasiSdkDownloadTempDir>$([System.IO.Path]::Combine($([System.IO.Path]::GetTempPath()), $([System.IO.Path]::GetRandomFileName())))</WasiSdkDownloadTempDir>
83-
</PropertyGroup>
84-
85-
<MakeDir Directories="$(WasiSdkDownloadTempDir)" />
86-
<DownloadFile
87-
SourceUrl="$(WasiSdkUrl)"
88-
DestinationFolder="$(WasiSdkDownloadTempDir)">
89-
<Output TaskParameter="DownloadedFile" ItemName="WasiSdkDownloadTempFile" />
90-
</DownloadFile>
91-
92-
<!-- Windows 10+ has tar built in, so this should work cross-platform -->
93-
<Message Importance="high" Text="Extracting @(WasiSdkDownloadTempFile) to $(WasiSdkRoot)..." />
94-
<MakeDir Directories="$(WasiSdkRoot)" />
95-
<Exec Command="tar -xf &quot;@(WasiSdkDownloadTempFile)&quot; -C . --strip-components=1" WorkingDirectory="$(WasiSdkRoot)" />
96-
<RemoveDir Directories="$(WasiSdkDownloadTempDir)" />
9722
</Target>
9823
</Project>

‎src/WitBindgen/WitBindgen.csproj

+9-5
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,14 @@
99

1010
<!-- Things you might want to edit -->
1111
<!-- Set BuildWitBindgenLocally to true if you want to build modules/wit-bindgen locally and use its output -->
12-
<BuildWitBindgenLocally>true</BuildWitBindgenLocally>
13-
<PrebuiltWitBindgenVersion>0.14.0</PrebuiltWitBindgenVersion>
14-
<PrebuiltWitBindgenBaseUrl>https://github.com/bytecodealliance/wit-bindgen/releases/download/wit-bindgen-cli-$(PrebuiltWitBindgenVersion)/wit-bindgen-v$(PrebuiltWitBindgenVersion)</PrebuiltWitBindgenBaseUrl>
15-
12+
<BuildWitBindgenLocally>false</BuildWitBindgenLocally>
13+
<PrebuiltWitBindgenVersion>0.26.0</PrebuiltWitBindgenVersion>
14+
<PrebuiltWitBindgenBaseUrl>https://github.com/bytecodealliance/wit-bindgen/releases/download/v$(PrebuiltWitBindgenVersion)/wit-bindgen-$(PrebuiltWitBindgenVersion)</PrebuiltWitBindgenBaseUrl>
1615
<WitBindgenModuleRoot>$(MSBuildThisFileDirectory)..\..\modules\wit-bindgen\</WitBindgenModuleRoot>
1716

17+
<!-- This is a marker file that lets the build scripts identity if the files need to be modified when updating versions -->
18+
<CurrentWitBindgenVersion>$(MSBuildThisFileDirectory)tools\version-$(PrebuiltWitBindgenVersion)</CurrentWitBindgenVersion>
19+
1820
<!-- Don't pack any assemblies in lib/*/.dll.-->
1921
<NoPackageAnalysis>true</NoPackageAnalysis>
2022
<IncludeBuildOutput>false</IncludeBuildOutput>
@@ -50,8 +52,10 @@
5052
</ItemGroup>
5153
</Target>
5254

53-
<Target Name="DownloadNativeToolingCore" Inputs="@(PrebuiltWitBindgenOutputs)" Outputs="@(PrebuiltWitBindgenOutputs)">
55+
<Target Name="DownloadNativeToolingCore" Condition="!Exists('$(CurrentWitBindgenVersion)')" Inputs="@(PrebuiltWitBindgenOutputs);$(CurrentWitBindgenVersion)" Outputs="@(PrebuiltWitBindgenOutputs);$(CurrentWitBindgenVersion)">
56+
<RemoveDir Directories="$(MSBuildThisFileDirectory)tools" />
5457
<DownloadFile SourceUrl="$(PrebuiltWitBindgenBaseUrl)-%(PrebuiltWitBindgenToolTarget.Identity)%(PrebuiltWitBindgenToolTarget.Ext)" DestinationFolder="tools\temp" DestinationFileName="%(PrebuiltWitBindgenToolTarget.Rid)%(PrebuiltWitBindgenToolTarget.Ext)" />
58+
<WriteLinesToFile File="$(CurrentWitBindgenVersion)" Lines="$(PrebuiltWitBindgenVersion)" Overwrite="true" WriteOnlyWhenDifferent="true" />
5559
<MakeDir Directories="tools\%(PrebuiltWitBindgenToolTarget.Rid)" />
5660
<Exec Command="tar -xf &quot;temp/%(PrebuiltWitBindgenToolTarget.Rid)%(PrebuiltWitBindgenToolTarget.Ext)&quot; -C %(PrebuiltWitBindgenToolTarget.Rid) --strip-components=1" WorkingDirectory="tools" />
5761
<RemoveDir Directories="tools\temp" />

0 commit comments

Comments
 (0)