Skip to content

Latest commit

 

History

History
853 lines (540 loc) · 27.1 KB

File metadata and controls

853 lines (540 loc) · 27.1 KB

How to use std properly including all the commands that are not in the prelude

discord

scope modules | where name =~ '^std' | get submodules.0

Powershell Reference on Arrays and Ranges

  1. cargo install samply
  2. samply record nu -n
  3. type exit to exit nushell
    ~/src/forks/nushell> exit
    Lost 10 events!
    Local server listening at http://127.0.0.1:3000
    Press Ctrl+C to stop.

and it automatically launches the screenshot website

Parser is called twice every time because of eval_hook

Start up nushell with a log level

nu --log-level info

Environment Variables in Nushell

To set an env

$env.IOX_DBNAME = 'company_sensors'

To get an env

$env | get IOX_DBNAME

Difference between echo and print

I've seen this a few times now. I'm not sure if my explanation is exact but I think about it like this. print puts output immediately. JT wrote this command one day when I was whining about not being able to debug scripts with echo. echo puts values in the pipeline, print's output does not go in the pipeline. Additionally, blocks will only output the last value, it apparently does not put multiple values in the pipeline.

It used to but this was changed here nushell/nushell#8292.

discord core

Move bytes commands back to core

discord 8/18/23

Blocks versus Closures

discord 7/20/23

Number of commands

427 commands without / nu-cmd-extra
472 commands with / nu-cmd-extra
548 commands with / nu-dataframe but not extra
593 commands with / nu-datraframe and nu-cmd-extra

all based on the number of commands in "help commands"

How to publish out new crates

nurun -n --no-std-lib
$nu.startup-time
nu --config $nu.config-path --env-config $nu.env-path -c "$nu.startup-time"
nu --no-std-lib -n -c "$nu.startup-time"
use std bench; bench { nu --config $nu.config-path --env-config $nu.env-path -c "$nu.startup-time" } --verbose

More details on startup options are here

apology in advance :)

  • some of the discord references in this document point to the core team channel; so unfortunately you will not be able to see those particular discord references...

3/10/2023 hooks and display output

an example of records

{a: 1, e: {b: 3, c: 5}, f: 7}

list list any

[[[1 2] [2 [4 4]] 1 2] [2 [4 4]]] | debug
[
[[1 2] [2 [4 4]] 1 2]
[2 [4 4]]
]
[[a b]; [1 2] [3 4] [5 6] [x y]] | get b.1 b.3 | to json

Kubouch great explanation of parsing

Would it help if you imagine parsing exactly as compiling? You could think of running Nushell code in similar terms as running Rust or C code: You compile source code to a machine code, then use a CPU hardware to run the compiled machine code and get some values back. In Nushell, instead of machine code, we compile (= parse) Nu language into a data structure (Expression, etc.). Then, we use the engine to evaluate the Expressions to produce Values. We're essentially a compiled language in the same sense as Rust or C but instead of compiling to assembly, we compile to our own intermediate representation. But we ultimately face the same limitations as the "traditional" languages.

discord reference

Discussion about nushell parser grammars which is not happening...

Also, we're currently experimenting with moving our syntax to a static grammar and trying out different syntax ideas: https://github.com/nushell/grammar. We're also preparing a rewrite of our parser once we have the grammar ready. One way that changed is also parsing the command calls: The parsing will happen the same way regardless of whether the command is extrernal or internal, but it would be the type checker that would match parsed command call to the expected signature. Therefore, at the output of the parser, we'll have all the arguments and their ordering, so we'd just need a mechanism to loosen up the rules in the type-checking stage and expose this information in the function body (maybe have a built-in $args variable?).

So, with the new way of thinking, the first part of "fall-through" signatures might come naturally, we just need to come up with a good way to do the second part (how to expose the args within the function).

source

In the past you could use the keyword source for scripts with custom commands, aliases, and environment variables

Moving forward this will no longer be the case.

  • source-env will be for environment variables which can now have dynamic paths
  • use will be for custom commands and aliases

Nulib Dirs

the default_env defines NU_LIB_DIRS which has a default "scripts" folder. If you put any nu script in there you can just source it by source name.nu

String Stuff

7z x ($env.SOME_ARCHIVE_PATH) -o($env.DEPLOY_DIR)
$"-o($env.DEPLOY_DIR)"

Nushell birthday

That’s correct. Public announcement was August 23rd, 2019 August 24th in New Zealand

discord

Exact number of stars

curl -s "https://api.github.com/repos/nushell/nushell" | grep stargazers_count | cut -d : -f 2 | tr -d " " | tr -d ","

Details about Environment Variables

discord

Fetching in Scripts

view-source stars
def stars [ --help (-h) ] {
    fetch https://api.github.com/repos/nushell/nushell | get stargazers_count
}

and another one...

fetch https://api.github.com/repos/nushell/nushell/releases/latest | get assets | select name download_count | sort-by download_count -r

$nu to see history-path and many others

$nu.history-path

the version command

TIL, if you want version to have the right commit_hash and build_time you need to touch crates\nu-cmd-lang\build.rs so this information gets rebuilt each time. discord

get versus select

get gets the information out of the structure. select maintains the structure. you can see this with ls | get name vs ls | select name

discord

Neat trick

def foo [] {
where size > 10kb
}

ls | foo

discord

More on $in

  • $in refers to the value passed from the pipe
  • in case of each $in represents each row

discord

let first = [[a b]; [1 2]]
let second = [[c d]; [3 4]]
let third = [[e f]; [5 6]]
[$first $second $third]|reduce {|it, acc| $acc|merge {$it}}

Transforming Records to Table

discord

Ansi Escapes

discordcore

Why do we need a $ sign for variables ?

someone asked: i dont really know what using let $bla brings when you could do let bla

@kubouch reply: This would confuse it with external commands. If Nushell sees a bare word (e.g., pwd) where it cannot be a string, it considers it an external command. Ditching $ would make bare words ambiguous: let x = pwd could mean both "run pwd command" and "fetch pwd variable". So I personally see $ useful.

JT response: Like @Kubouch says, having the $ makes variables unambiguous.

discord

Is there a nu equivalent for bash export BROWSER=w3m ?

My bad, you have to quote the value.
let-env BROWSER = "w3m"

discord

query db command

loving your query db command. i have a question though. i ran this command and it works as expected...

open ./cities.db | query db "select * from cities where country = 'New Zealand' order by city"

however if i do let db = (open ./cities.db) and then $db | query db "blah" it doesn't work.

do i need to open the file each time?

i mean, obviously, you do but what i'd like to do is store the open connection as above and just use it.

discord

how to output to /dev/null ?

| ignore and do -i {... } | ignore

Issue

How errors are displayed

3 / "bob"

Adding env.nu, why ?

Just for some context, if I understand correctly env.nu was added because we otherwise couldn't access the updated environment variables in config.nu (for env vars that were updated in config.nu). That meant that we couldn't use files in any of the NU_LIB_DIRS from config.nu, because that env var was defined in the same file. By first sourcing env.nu and defining NU_LIB_DIRS there, we can now use files from those directories in config.nu. Is that correct?

Evaluate parenthesis

discord

Export and use

in order to use blah.nu * you need to have exported defs as well like export def on the defs that you want to be able to call externally - if defs are called internally by other defs only, they don't need to be exported.

see nushell book on modules

discord

Info about decls, each taking a block etc...

discord

First cut at @elferherrera menuverse

discord core

In a block what are the captures ?

the variables outside of a block that it's using. So in:

let x = 10
do {
  let y = 20
  print ($x + $y)
}

then $x is a capture of the block given to do

discord core

Help command

  • ctrl q gets you going
  • hit tab to move around
  • option [up, down] etc...

git push versus git status, why are they different

discord

how escaping works in nushell

"one\ntwo"
'one\ntwo'

For more details on this topic see issue 4869

$nu is your gateway into lots of internal nushell stuff

When developing nushell, remember to...

Remember to blow away all of your old nushell processes everytime you do a code update and rebuild nushell otherwise you will get into the problem I was seeing this morning with history and probably other stuff

how the bang and history commands work

if you type history | last 10 and look the index columns - that's what you use with !number... if you're in ctrl-x history mode, you can search, and then just type !5 and it'll choose the 5th item, assuming you have it setup that way

discord core

key points

let list = help commands | select name # This breaks
let $list = (help commands | select name | first 10) # This works

building a string or concatenation

is build-str the only way to build a string? we don't have an append or something like that. right? you can also do "hello" + " world"

sourcing a file

discord

source requires a known string. it doesn't support dynamically creating a string. it's a design choice because: In Nushell, we're trying to make it so we know all of the source at "parse time". This lets us later add really good IDE support, and for Nushell to more easily scale up to large projects

keybindings

keybindings list --events
keybindings default

blocks and parentheses, what are they ?

discord

strings, nuon, escape characters philosophy

discord

path stuff

is there a replacement for pathvar in .59? let-env PATH = ($env.PATH | append foo)

$it notes

See the book section Working with lists

$in notes

when you want to just grab what the next to last pipeline is outputting and test it (for whatever reason)...

And $in is the variable that allows you to work with all of the data coming in from the pipeline in one place.

The $in variable will collect the pipeline into a value for you, allowing you to access the whole stream as a parameter.

"john ran to the store" | str length | $in > 25

For more details....

rg -F '$in'
tutor -f "$in"
tutor var

Alias Notes

alias "dfr describe" = dfr describe -q [0.5 0.90 0.95 0.99]

Env Notes

discord

Need to look into optional arguments

here is an example discord

more details discord

more research on nu as a language discord

First cut at exit codes

discord-core

and the PR

Value and Strings

discord-core

is there a difference between Value::as_string and Value::into_string?

as_string() converts any value that supports it to a string. into_string() formats the value and prints it, even lists etc. Maybe into_string() could be called print() or format() or something.

yeah, in theory there's a difference, so we don't accidentally convert something to string that shouldn't be but, like your external args, we should be able to safely convert ints to strings

There are two types of Streams in Nushell

  • ListStream
  • RawStream

ListStreams are used for Values any time you have an iterator

RawStreams are used for externals as well as the open command when you don't have a file extension that you know about. In other words, it goes to raw when you do not know what you are going to get back...

Both of these streams are referenced in nu-protocol in

  • pipeline_data.rs
  • value/stream.rs

what is the exempt label in issues

exempt from the stale bot

what is --testbin

--testbin String what does it mean to run internal test binary ?

It's a known external we can call from our tests, since not all platforms have echo, cat, etc...

ok so that means an end user would never use that feature (except if they were writing tests as a nushell developer for our code base)

discord core

changes in nushell from legacy nushell

  • pivot is now transpose
  • insert (is now part of update)
  • and nth (is now part of select)

Rolling towards release day

nushell/nushell#4300

nushell/nushell#4305

nushell/nushell#4314

nushell/nushell#4356

Location of the file config.nu

  • $nu.config-path
  • on mac: /Users/username/Library/Application Support/nushell

use metadata to create valid spans

discord

but these spans need to be valid spans for this to work, and you can't create valid spans yourself (you need to use helpers like metadata).

append command

To see raw input values print val in the append command

let val: Value = call.req(engine_state, stack, 0)?;
println!("{:?}",val);
let vec: Vec<Value> = process_value(val);

engine-q notes

How to run engine-q

If you want to see all the plugins you need to do

cargo build --features=extra
cargo run --features=extra

nurun --help

Json with no formatting chars

[[a b]; [jim susie] [3 4]] | to json | str find-replace '\n' '' -a | str trim -a

Send back no data

Ok(PipelineData::Value(Value::Nothing { span: call.head }, None,))

// Or this way if Span is passed into the function

Ok(PipelineData::Value(Value::Nothing { span: *span }, None))

trait CustomValue

discord

Getting column data

  • get_data_by_key

commands that use get_data_by_key include empty? and compact

If you review empty? and debug the code you will see that given a particular column name

for column in column_paths.clone() {
    let path = column.into_string();
    let data = input.get_data_by_key(&path);
    println!("{:?} {:?}",path,data);

It goes through and prints each value going down the column...

  • follow_cell_path
PathMember::Int
PathMember::String

The String arguments to follow_cell_path are for the column names. The Int arguments to follow_cell_path are for the rows.

[[a,b];[rick,pete], [bill,paul]] | get a
[[a,b];[rick,pete], [bill,paul]] | get b
[[a,b];[rick,pete], [bill,paul]] | get 0
[[a,b];[rick,pete], [bill,paul]] | get 1

For more details...

rg follow_cell_path
fn run(
    &self,
    engine_state: &EngineState,
    stack: &mut Stack,
    call: &Call,
    input: PipelineData,
) -> Result<nu_protocol::PipelineData, nu_protocol::ShellError> {
    let columns: Vec<CellPath> = call.rest(engine_state, stack, 0)?;
    let span = call.head;

In lots of commands you see the "call.rest" feature, this functionality is defined in the nu-engine crate in call_ext

pub trait CallExt {

Random commands

[[name,age];[bill,20],[rick,21]] | append [[name,age]; [paul, 40], [hb,70], [sally, 33], [sam,46]] | prepend [[name,age]; [sarah,40],[jane,50]]

How do you run CI checks locally ?

alias ciman='cargo fmt; cargo check; cargo clippy; cargo test --all'

cargo fmt
cargo check
cargo clippy
cargo test --all

CI workflow steps manually;    discord link

Other useful clippy commands

cargo clippy -- -W clippy::pedantic

Rust Embed

discord

we use this thing called rust-embed to embed things into the binary. Right now it looks like we're embedding themes for doing html, but I think you can use it for this as well

General Questions

what is our convention about commands having underscores ?

It's because commands like move_ is also a Rust keyword.

So commands like let, do, if have underscore, otherwise when you want to include them as a module, you'd type mod if and the compiler would complain that you cannot put if after mod.

Code Questions

example 24-bit terminals on mac including kitty, tabby, iterm2, the default mac terminal is not 24-bit.

Anyone has an idea how to check if a value's type is Record?

You can pattern-match on the value:

if let Value::Record { .. } = value { <code> }

Convert tables and rows to Strings

Does anyone know offhand if there is code in nushell to convert tables and rows to Strings? I know there is the autoview command, but it uses a bunch of config stuff, etc. which I am trying to avoid

discord

the engine-q table command will convert to strings in nushell, viewers don't return anything

getting columns names in engine-q similar to nushell

discord

Do not use...

Span::unknown()

As discussed here in PR 242

When working on commands, you can use call.head (this is a good option for new values created within the command) or reuse spans that come with Values from the input stream or command arguments. To get it right requires a bit of playing around so a good idea is to also purposefully trigger the errors and see how the messages look like.

References

Difference between a shell and a scripting language

discord

  • nakst Philosophically, what is the difference between a shell and a scripting language REPL? (Sorry if this is the wrong place to ask) My intuition is that the former is designed for programmatic coordination of other applications, while the latter is instead aimed primarily at computation. But I'm interested to hear what people who have spent more time thinking about shells would argue.

  • jt for nushell, we're trying to merge the two concepts into one traditionally a shell would be for interacting with the system directly and a REPL would be for interacting with the language's engine directly. For us, we'd like to do both equally well

  • nakst yeah, I'm trying to make a REPL for my scripting language but give it some shell-like capabilities I find a little difficult to strike the right balance between the two

How to get input from the user ?

See the input command.

Lots of details about events and how order matters

discord start here, coreteam

Where does Nushell come from

"it's a pun on new, but also comes from hebrew/yiddish (originally named by Yehuda)"

Move nth to select PR

This moves nth into select. This works by looking at the cell path we're given. If the cell path is a number, we follow the same logic as get: instead of a column name, use this as a row number.

The end result is that now select works like get, but instead of extracting data, it down-selects data and keeps the original shape intact. I think this will help teaching, as you can remember that one commands down-selects and one extracts and that works either for colum...

ref: #4385

When do I use Snake and when do I use Kebab ?

We know we didn't like either extreme, so after chatting with folks I think we should have a balance between the two: Here's my proposal:

Snake:

  • Column names
  • Cell paths (cell paths are column names)
  • Record fields (record fields are column names)
  • Variables (less confusing if you have math, eg) a_b - c_d is easier than a-b - c-d
  • env vars

Kebab:

  • Command names
  • Subcommand names
  • Flags (kebab flags appear to be the standard for most of the apps I checked)

discord

How do I set up custom prompts ?

discord

Long flags do not include "="

discord

How do I build up a string ?

let age = 10
echo "my age is " $age | str collect

Other useful Cargo commands

cargo update --package reedline

More details on cargo update

nushell/nushell#5184

Install cargo-outdated as a binary just like you do rg, or whalespotter

Then to run the command go to a particular crate in nushell and run...

cargo outdated -R

Find unused dependencies

cargo +nightly udeps --all-targets

prql

open movies3.csv |
select LeadStudio WorldwideGross |
group-by LeadStudio |
transpose company gross |
insert total {
|g| $g.gross |
reduce -f 0 {|i acc| $acc + $i.WorldwideGross}} |
reject gross |
sort-by total

discord

Cargo commands

Relevant cargo commands to track down dependencies and get rid of duplicated crates. @sholderbach mentioned these two commands in our core team meeting.

cargo tree -- duplicated
cargo build -- timings

Nushell Language Search on Github