Skip to content

Improve error reporting when loading the Docker configuration file #1263

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 20 additions & 13 deletions src/Testcontainers/Builders/DockerConfig.cs
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ public JsonDocument Parse()
/// Executes a command equivalent to <c>docker context inspect --format {{.Endpoints.docker.Host}}</c>.
/// </remarks>
/// A <see cref="Uri" /> representing the current Docker endpoint if available; otherwise, <c>null</c>.
[CanBeNull]
[NotNull]
public Uri GetCurrentEndpoint()
{
const string defaultDockerContext = "default";
Expand All @@ -99,16 +99,27 @@ public Uri GetCurrentEndpoint()
var dockerContextHash = BitConverter.ToString(sha256.ComputeHash(Encoding.Default.GetBytes(dockerContext))).Replace("-", string.Empty).ToLowerInvariant();
var metaFilePath = Path.Combine(_dockerConfigDirectoryPath, "contexts", "meta", dockerContextHash, "meta.json");

if (!File.Exists(metaFilePath))
try
{
return null;
using (var metaFileStream = File.OpenRead(metaFilePath))
{
var meta = JsonSerializer.Deserialize(metaFileStream, SourceGenerationContext.Default.DockerContextMeta);
var host = meta.Endpoints?.Docker?.Host;
if (host == null)
{
throw new DockerConfigurationException($"The Docker host is null in {metaFilePath} (JSONPath: Endpoints.docker.Host)");
}

return new Uri(host.Replace("npipe:////./", "npipe://./"));
}
}

using (var metaFileStream = File.OpenRead(metaFilePath))
catch (Exception notFoundException) when (notFoundException is DirectoryNotFoundException or FileNotFoundException)
{
var meta = JsonSerializer.Deserialize(metaFileStream, SourceGenerationContext.Default.DockerContextMeta);
var host = meta?.Name == dockerContext ? meta.Endpoints?.Docker?.Host : null;
return string.IsNullOrEmpty(host) ? null : new Uri(host.Replace("npipe:////./", "npipe://./"));
throw new DockerConfigurationException($"The Docker context '{dockerContext}' does not exist", notFoundException);
}
catch (Exception exception) when (exception is not DockerConfigurationException)
{
throw new DockerConfigurationException($"The Docker context '{dockerContext}' failed to load from {metaFilePath}", exception);
}
Comment on lines -107 to 123
Copy link
Collaborator

@HofmeisterAn HofmeisterAn Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to explain my concerns. Maybe this is hypothetical, but let's assume:

  • The ~/.docker/config.json file exists.
  • The file contains a currentContext node.
  • The meta directory or file does not exist, or the currentContext cannot be found.
  • The user runs a remote container runtime, e.g. configured with WithDockerEndpoint(Uri).

With this implementation, an exception will be thrown when creating the list of Docker endpoints to check for availability.

System.TypeInitializationException
The type initializer for 'DotNet.Testcontainers.Configurations.TestcontainersSettings' threw an exception.

DotNet.Testcontainers.Builders.DockerConfigurationException
The Docker context '...' does not exist

System.IO.DirectoryNotFoundException
Could not find a part of the path '...'.

This prevents users from overriding the builder configuration using the API mentioned above. That said, I don't think WithDockerEndpoint(Uri) is the real issue here. If a user falls back to this API by default, something else is probably wrong.

My main concern is that this exception could also happen in other configurations. I believe this was one of the reasons we didn't throw exceptions in the initial implementation of Docker context support (IIRC, we removed it on purpose). Until now, we've been failing silently allowing another provider to find an available Docker endpoint.

The DockerConfigTests were updated and a new ThrowsWhenDockerConfigEndpointNotFound test was added.

I definitely agree with you on this. It bothers me too. I don't have a strong opinion on this PR. I was considering adding more logging to show which provider is processed and chosen (and which ones aren't). But if you don't see any issues with throwing an exception, we can go ahead and merge it. LMKWYT.

Copy link
Contributor Author

@0xced 0xced Apr 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it's a bit unfortunate to throw if a valid endpoint is configured through WithDockerEndpoint(Uri). On the other hand, not throwing in this case makes it really hard to diagnose a broken Docker Desktop configuration.

Also, this case should really be exceptional and I think it warrants an exception that must be addressed by actually fixing the Docker Desktop configuration.

There's also this important change in the DockerDesktopEndpointAuthenticationProvider:

diff --git b/src/Testcontainers/Builders/DockerDesktopEndpointAuthenticationProvider.cs a/src/Testcontainers/Builders/DockerDesktopEndpointAuthenticationProvider.cs
index f2363f89..e1352585 100644
--- b/src/Testcontainers/Builders/DockerDesktopEndpointAuthenticationProvider.cs
+++ a/src/Testcontainers/Builders/DockerDesktopEndpointAuthenticationProvider.cs
@@ -15,7 +15,7 @@ namespace DotNet.Testcontainers.Builders
     /// Initializes a new instance of the <see cref="DockerDesktopEndpointAuthenticationProvider" /> class.
     /// </summary>
     public DockerDesktopEndpointAuthenticationProvider()
-      : base(DockerConfig.Instance.GetCurrentEndpoint()?.AbsolutePath, GetSocketPathFromHomeDesktopDir(), GetSocketPathFromHomeRunDir())
+      : base(DockerConfig.Instance.GetCurrentEndpoint())
     {
     }

It means that Testcontainers won't blindly try to connect to potentially valid sockets (from home desktop or home run directories). I've experienced issues doing so in the past where the socket was existing and but connecting to the socket would just stall because the engine was not running. It was very hard to diagnose!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Sorry for the delay. I will try to merge both PRs by the end of this week (I will add both to 4.5.0).

}
}
Expand Down Expand Up @@ -162,15 +173,11 @@ private string GetDockerContext()
internal sealed class DockerContextMeta
{
[JsonConstructor]
public DockerContextMeta(string name, DockerContextMetaEndpoints endpoints)
public DockerContextMeta(DockerContextMetaEndpoints endpoints)
{
Name = name;
Endpoints = endpoints;
}

[JsonPropertyName("Name")]
public string Name { get; }

[JsonPropertyName("Endpoints")]
public DockerContextMetaEndpoints Endpoints { get; }
}
Expand Down
29 changes: 29 additions & 0 deletions src/Testcontainers/Builders/DockerConfigurationException.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
namespace DotNet.Testcontainers.Builders
{
using System;
using JetBrains.Annotations;

/// <summary>
/// The exception that is thrown when the Docker configuration file cannot be read successfully.
/// </summary>
[PublicAPI]
public sealed class DockerConfigurationException : Exception
{
/// <summary>
/// Initializes a new instance of the <see cref="DockerConfigurationException"/> class, using the provided message.
/// </summary>
/// <param name="message">The error message that explains the reason for the exception.</param>
public DockerConfigurationException(string message) : base(message)
{
}

/// <summary>
/// Initializes a new instance of the <see cref="DockerConfigurationException"/> class, using the provided message and exception that is the cause of this exception.
/// </summary>
/// <param name="message">The error message that explains the reason for the exception.</param>
/// <param name="innerException">The exception that is the cause of the current exception.</param>
public DockerConfigurationException(string message, Exception innerException) : base(message, innerException)
{
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ internal sealed class DockerDesktopEndpointAuthenticationProvider : RootlessUnix
/// Initializes a new instance of the <see cref="DockerDesktopEndpointAuthenticationProvider" /> class.
/// </summary>
public DockerDesktopEndpointAuthenticationProvider()
: base(DockerConfig.Instance.GetCurrentEndpoint()?.AbsolutePath, GetSocketPathFromHomeDesktopDir(), GetSocketPathFromHomeRunDir())
: base(DockerConfig.Instance.GetCurrentEndpoint())
{
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,15 @@ public RootlessUnixEndpointAuthenticationProvider(params string[] socketPaths)
DockerEngine = socketPath == null ? null : new Uri("unix://" + socketPath);
}

/// <summary>
/// Initializes a new instance of the <see cref="RootlessUnixEndpointAuthenticationProvider" /> class.
/// </summary>
/// <param name="dockerEngine">The Unix socket Docker Engine endpoint.</param>
public RootlessUnixEndpointAuthenticationProvider(Uri dockerEngine)
{
DockerEngine = dockerEngine;
}

/// <summary>
/// Gets the Unix socket Docker Engine endpoint.
/// </summary>
Expand Down
66 changes: 46 additions & 20 deletions tests/Testcontainers.Tests/Unit/Builders/DockerConfigTest.cs
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ public void ReturnsDefaultEndpointWhenDockerContextIsDefault()
public void ReturnsConfiguredEndpointWhenDockerContextIsCustomFromPropertiesFile()
{
// Given
using var context = new ConfigMetaFile("custom", "tcp://127.0.0.1:2375/");
using var context = new ConfigMetaFile("custom", new Uri("tcp://127.0.0.1:2375/"));

ICustomConfiguration customConfiguration = new PropertiesFileConfiguration(new[] { "docker.context=custom", context.GetDockerConfig() });
ICustomConfiguration customConfiguration = new PropertiesFileConfiguration(new[] { "docker.context=custom", $"docker.config={context.DockerConfigDirectoryPath}" });
var dockerConfig = new DockerConfig(customConfiguration);

// When
Expand All @@ -62,10 +62,10 @@ public void ReturnsConfiguredEndpointWhenDockerContextIsCustomFromPropertiesFile
public void ReturnsConfiguredEndpointWhenDockerContextIsCustomFromConfigFile()
{
// Given
using var context = new ConfigMetaFile("custom", "tcp://127.0.0.1:2375/");
using var context = new ConfigMetaFile("custom", new Uri("tcp://127.0.0.1:2375/"));

// This test reads the current context JSON node from the Docker config file.
ICustomConfiguration customConfiguration = new PropertiesFileConfiguration(new[] { context.GetDockerConfig() });
ICustomConfiguration customConfiguration = new PropertiesFileConfiguration(new[] { $"docker.config={context.DockerConfigDirectoryPath}" });
var dockerConfig = new DockerConfig(customConfiguration);

// When
Expand All @@ -83,17 +83,37 @@ public void ReturnsActiveEndpointWhenDockerContextIsUnset()
}

[Fact]
public void ReturnsNullWhenDockerContextNotFound()
public void ThrowsWhenDockerContextNotFound()
{
// Given
ICustomConfiguration customConfiguration = new PropertiesFileConfiguration(new[] { "docker.context=missing" });
var dockerConfig = new DockerConfig(customConfiguration);

// When
var currentEndpoint = dockerConfig.GetCurrentEndpoint();
var exception = Assert.Throws<DockerConfigurationException>(() => dockerConfig.GetCurrentEndpoint());

// Then
Assert.Null(currentEndpoint);
Assert.Equal("The Docker context 'missing' does not exist", exception.Message);
Assert.IsType<DirectoryNotFoundException>(exception.InnerException);
}

[Fact]
public void ThrowsWhenDockerConfigEndpointNotFound()
{
// Given
using var context = new ConfigMetaFile("custom");

ICustomConfiguration customConfiguration = new PropertiesFileConfiguration(new[] { "docker.context=custom", $"docker.config={context.DockerConfigDirectoryPath}" });
var dockerConfig = new DockerConfig(customConfiguration);

// When
var exception = Assert.Throws<DockerConfigurationException>(() => dockerConfig.GetCurrentEndpoint());

// Then
Assert.StartsWith("The Docker host is null in ", exception.Message);
Assert.Contains(context.DockerConfigDirectoryPath, exception.Message);
Assert.EndsWith(" (JSONPath: Endpoints.docker.Host)", exception.Message);
Assert.Null(exception.InnerException);
}
}

Expand All @@ -117,9 +137,9 @@ public void ReturnsActiveEndpointWhenDockerHostIsEmpty()
public void ReturnsConfiguredEndpointWhenDockerHostIsSet()
{
// Given
using var context = new ConfigMetaFile("custom", "");
using var context = new ConfigMetaFile("custom");

ICustomConfiguration customConfiguration = new PropertiesFileConfiguration(new[] { "docker.host=tcp://127.0.0.1:2375/", context.GetDockerConfig() });
ICustomConfiguration customConfiguration = new PropertiesFileConfiguration(new[] { "docker.host=tcp://127.0.0.1:2375/", $"docker.config={context.DockerConfigDirectoryPath}" });
var dockerConfig = new DockerConfig(customConfiguration);

// When
Expand Down Expand Up @@ -147,26 +167,32 @@ private sealed class ConfigMetaFile : IDisposable

private const string MetaFileJson = "{{\"Name\":\"{0}\",\"Metadata\":{{}},\"Endpoints\":{{\"docker\":{{\"Host\":\"{1}\",\"SkipTLSVerify\":false}}}}}}";

private readonly string _dockerConfigDirectoryPath;
public string DockerConfigDirectoryPath { get; }

public ConfigMetaFile(string context, string endpoint, [CallerMemberName] string caller = "")
public ConfigMetaFile(string context, [CallerMemberName] string caller = "")
{
_dockerConfigDirectoryPath = Path.Combine(TestSession.TempDirectoryPath, caller);
var dockerContextHash = Convert.ToHexString(SHA256.HashData(Encoding.Default.GetBytes(context))).ToLowerInvariant();
var dockerContextMetaDirectoryPath = Path.Combine(_dockerConfigDirectoryPath, "contexts", "meta", dockerContextHash);
_ = Directory.CreateDirectory(dockerContextMetaDirectoryPath);
File.WriteAllText(Path.Combine(_dockerConfigDirectoryPath, "config.json"), string.Format(ConfigFileJson, context));
File.WriteAllText(Path.Combine(dockerContextMetaDirectoryPath, "meta.json"), string.Format(MetaFileJson, context, endpoint));
DockerConfigDirectoryPath = InitializeContext(context, null, caller);
}

public string GetDockerConfig()
public ConfigMetaFile(string context, Uri endpoint, [CallerMemberName] string caller = "")
{
return "docker.config=" + _dockerConfigDirectoryPath;
DockerConfigDirectoryPath = InitializeContext(context, endpoint, caller);
}

private static string InitializeContext(string context, Uri endpoint, [CallerMemberName] string caller = "")
{
var dockerConfigDirectoryPath = Path.Combine(TestSession.TempDirectoryPath, caller);
var dockerContextHash = Convert.ToHexString(SHA256.HashData(Encoding.Default.GetBytes(context))).ToLowerInvariant();
var dockerContextMetaDirectoryPath = Path.Combine(dockerConfigDirectoryPath, "contexts", "meta", dockerContextHash);
_ = Directory.CreateDirectory(dockerContextMetaDirectoryPath);
File.WriteAllText(Path.Combine(dockerConfigDirectoryPath, "config.json"), string.Format(ConfigFileJson, context));
File.WriteAllText(Path.Combine(dockerContextMetaDirectoryPath, "meta.json"), endpoint == null ? "{}" : string.Format(MetaFileJson, context, endpoint.AbsoluteUri));
return dockerConfigDirectoryPath;
}

public void Dispose()
{
Directory.Delete(_dockerConfigDirectoryPath, true);
Directory.Delete(DockerConfigDirectoryPath, true);
}
}
}
Expand Down