Skip to content

Deploy notebooks from CLI - #3034

Merged
aleDsz merged 14 commits into
mainfrom
ale-deploy-from-cli
Jul 24, 2025
Merged

Deploy notebooks from CLI#3034
aleDsz merged 14 commits into
mainfrom
ale-deploy-from-cli

Conversation

@aleDsz

@aleDsz aleDsz commented Jul 17, 2025

Copy link
Copy Markdown
Member

The status feedback message is clunky, but the deploy feature is working.

Kooha-2025-07-24-13-28-02.mp4

@aleDsz aleDsz self-assigned this Jul 17, 2025
Comment thread lib/livebook/migration.ex Outdated
Comment thread lib/livebook_cli.ex Outdated
Comment thread lib/livebook_cli.ex Outdated
Comment thread lib/livebook_cli/deploy.ex Outdated
Comment thread lib/livebook_cli/task.ex
Comment thread lib/livebook_cli/utils.ex Outdated
Comment thread lib/livebook/application.ex Outdated
Comment thread lib/livebook/teams/requests.ex Outdated
@aleDsz
aleDsz force-pushed the ale-deploy-from-cli branch from 5b3ae00 to cb6594a Compare July 23, 2025 21:31
@aleDsz
aleDsz force-pushed the ale-deploy-from-cli branch from cb6594a to ecfa697 Compare July 23, 2025 21:32
@aleDsz
aleDsz requested a review from josevalim July 23, 2025 21:56
Comment thread lib/livebook/migration.ex Outdated
Comment thread lib/livebook/teams.ex Outdated
Comment thread lib/livebook/teams/requests.ex Outdated
Comment thread lib/livebook_cli.ex Outdated
Comment thread lib/livebook_cli/task.ex Outdated
"""
end

defp format_exception(%RuntimeError{} = exception, _, _) do

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately regular Elixir code can also emit RuntimeError, and it would get mixed up with this. If you want this functionality, I would suggest defning a LivebookCLI.Error and raise that instead.

Comment thread lib/livebook_cli/deploy.ex Outdated
Comment thread lib/livebook_cli/deploy.ex Outdated
Comment thread lib/livebook_cli/deploy.ex Outdated
Comment thread lib/livebook_cli.ex Outdated
Comment thread lib/livebook_cli/deploy.ex Outdated
Comment on lines +7 to +8
@deploy_key_prefix Teams.Requests.deploy_key_prefix()
@teams_key_prefix Teams.Org.teams_key_prefix()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if we could have these in a single place. What about having a Teams.Prefixes module or Teams.Constants module which only has this stuff? This will also help keep compile cycles small, as those files won't have any logic in them.

Comment thread lib/livebook_cli/deploy.ex Outdated
Comment on lines +156 to +160
path
|> File.ls!()
|> Enum.map(&Path.join(path, &1))
|> Enum.reject(&File.dir?/1)
|> Enum.filter(&String.ends_with?(&1, ".livemd"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
path
|> File.ls!()
|> Enum.map(&Path.join(path, &1))
|> Enum.reject(&File.dir?/1)
|> Enum.filter(&String.ends_with?(&1, ".livemd"))
path
|> Path.join("*.livemd")
|> Path.wildcard()
|> Enum.filter(&File.regular?/1)

However, should we deploy only the direct files in a directory or all files recursively? Another option is to not support directories and instead allow a list of files to be given. This way the user can write this:

livebook deploy path/*.livemd

The shell will expand *.livemd into multiple entries and we just deploy all of them. WDYT?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR for LIVEBOOK_APPS_PATH we do path/**/*.livemd:

pattern = Path.join([dir, "**", "*.livemd"])
for path <- Path.wildcard(pattern) do
markdown = File.read!(path)

But passing multiple files sounds good to me.

Comment thread lib/livebook/storage.ex Outdated
if persist_storage? do
load_or_create_table()
else
:ets.new(__MODULE__, [:protected, :duplicate_bag])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently we duplicate :ets.new(__MODULE__, [:protected, :duplicate_bag]) in two places. Let's pass persist_storage? to load_or_create_table instead and have the condition there, such that we create the table in a single place only.

"""
@spec fetch_cli_session(map()) :: api_result()
def fetch_cli_session(config) do
post("/api/v1/cli/auth", %{}, config)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We call this with config = %Team{}, do we actually want to send all the fields?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To call this function, we don't really use the %Team{} struct, we send a map to fetch some data from Teams, so then we can build the %Team{} struct.

Comment thread lib/livebook_cli/deploy.ex Outdated
Comment thread lib/livebook_cli/deploy.ex Outdated
Comment thread lib/livebook_cli/deploy.ex Outdated
Comment on lines +156 to +160
path
|> File.ls!()
|> Enum.map(&Path.join(path, &1))
|> Enum.reject(&File.dir?/1)
|> Enum.filter(&String.ends_with?(&1, ".livemd"))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR for LIVEBOOK_APPS_PATH we do path/**/*.livemd:

pattern = Path.join([dir, "**", "*.livemd"])
for path <- Path.wildcard(pattern) do
markdown = File.read!(path)

But passing multiple files sounds good to me.

Comment thread lib/livebook_cli/deploy.ex Outdated
Comment thread lib/livebook_cli/utils.ex Outdated
Comment thread lib/livebook_cli/utils.ex Outdated
Comment thread lib/livebook_cli/deploy.ex Outdated
"""

{:transport_error, reason} ->
print_text([:red, " * #{app_deployment.title} failed to deploy."])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels we are a bit inconsistent. We use print_text([:red, ...]) but we also have log_error. Perhaps we should remove print_text and use either log_error (which would be moved to Utils)orlog_infoorlog_debugorlog_warn`.

@josevalim josevalim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dropped some small nits but otherwise this is good to go!

@aleDsz
aleDsz merged commit 4997e07 into main Jul 24, 2025
4 checks passed

@jonatanklosko jonatanklosko left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic!

@hugobarauna
hugobarauna deleted the ale-deploy-from-cli branch August 18, 2025 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants