Skip to content

repl-remote.md

Latest commit

 

History

History
591 lines (412 loc) · 13.3 KB

repl-remote.md

File metadata and controls

591 lines (412 loc) · 13.3 KB
Raw

Remote REPL

The remote REPL is a Venice network based REPL that provides a REPL server and client for interactive inspection and control of remote Venice applications.

The communication between the local REPL and the remote REPL server is transparently AES-256 GCM (AES Galois Counter Mode) encrypted. The secret for establishing an AES-256 encryption is generated by the Diffie-Hellman key exchange algorithm.

Venice applications are not aware of the REPL server. No specific preparation is required.

   

   

Usage

  • Start Venice application with a REPL server attached
  • Connect to the server using the standard Venice REPL
  • Start inspection and control of the remote Venice application!

 

Start Venice Application with REPL server

To attach a REPL server to a Venice application just add two launch parameters to set a port and a password:

Option Mandatory Default Description Example
-repl-port {port} yes - REPL communication port -repl-port 33334
-repl-pwd {password} yes - REPL password -repl-pwd xcf6zu=UI
-repl-encrypt {on-off} no on REPL transport encryption -repl-encrypt off
-repl-compress {on-off} no off REPL transport compression -repl-compress on
-repl-session-timeout t no 30 REPL session inactivity timeout in minutes -repl-session-timeout 15

 

If encryption is enabled all data exchanged between the client and the server REPL is AES-256-GCM encrypted. The secret key for the symmetric AES-256-GCM encryption is created and exchanged using the Diffie-Hellman key exchange algorithm.

 

Start the application with the REPL server port and password:

java -jar ./venice-1.13.10.jar -repl-port 33334 -repl-pwd 123 -file ./remote-repl-demo.venice

 

The remote REPL demo application looks like:

(do
  (ns demo)

  (def- stop? (atom false))


  (defn stop [] 
    (reset! stop? true)
    (println "Stopping demo server..."))

  (defn sum [x y] 
    (println "Function 'sum' called")
    (+ x y))


  (println "Started demo server")

  ;; just sleep until we get stopped
  (while (not @stop?)
    (sleep 1000))

  (println "Stopped demo server"))

It offers two public functions that can be remotely called:

Function Description Example
demo/sum sum two numbers (demo/sum 2 3)
demo/stop stop the application (demo/stop)

 

Connect from a REPL

Start a REPL and connect to the remote REPL server (pass host, port, and password)

venice> !remote localhost 33334 123

remote>

check the remote REPL Venice version

remote> (version)
=> 1.12.82

call a remote function

remote> (demo/sum 3 4)
Function 'sum' called
=> 7

switch back to the local REPL environment

remote> !local

venice>

 

Remote REPL demo session

The demo uses the remote REPL demo application listed above.

 

Standard out/err text produced by the remote functions are routed back to the REPL client and displayed in the according REPL colors.

REPL (client) Remote Application

   

Tracing via remote REPL

The remote REPL allows the ad hoc tracing of functions in the remote application. The tracing can be turned on and off for any public function in the remote application.

A function under tracing behaves as the original function. Venice just adds an interceptor that wraps the original function. Tracing interceptors can be dynamically added and removed.

A trace prints the arguments when the function is entered and the value of the function when it exits; indentation shows the level of recursive nesting.

Trace actions:

  • prints the function name and its evaluated arguments
  • calls the original function
  • prints the original function's return value
  • returns the original function's return value

 

Note

Tracing via remote REPL requires Venice 1.12.83+

 

The remote REPL demo application looks like:

(do
  (ns demo)

  (def- stop? (atom false))

  (defn stop [] 
    (reset! stop? true)
    (println "Stopping demo server..."))


  ;; trace demo functions
  (defn foo [x] (+ x 2))
  (defn zoo [x] (foo x))
  (defn bar [x] (zoo x))
  (defn foo-ex [x] (/ x 0)) ;; division by zero!
  (defn bar-ex [x] (foo-ex x))
  (defn factorial [n] (if (= n 0) 1 (* n (factorial (- n 1)))))


  (println "Started demo server")

  ;; just sleep until we get stopped
  (while (not @stop?)
    (sleep 1000))

  (println "Stopped demo server"))

 

i) Start the remote application

java -jar ./venice-1.13.10.jar -repl-port 33334 -repl-pwd 123 -file ./remote-repl-trace-demo.venice

ii) Connect from a REPL

Start a REPL and connect to the remote REPL server (pass host, port, and password) and load the tracing module (this module is loaded into the server application!).

venice> !remote localhost 33334 123

remote> (load-module :trace)

iii) Enable demo app functions for tracing

remote> (trace/trace-var +)
remote> (trace/trace-var demo/foo)
remote> (trace/trace-var demo/zoo)
remote> (trace/trace-var demo/bar)

remote> (trace/trace-var /)
remote> (trace/trace-var demo/foo-ex)
remote> (trace/trace-var demo/bar-ex)

remote> (trace/trace-var demo/factorial)

iv) Call a remote function

Example 1

remote> (demo/bar 5)

output:

TRACE t18: (demo/bar 5)
TRACE t19: | (demo/zoo 5)
TRACE t20: | | (demo/foo 5)
TRACE t21: | | | (core/+ 5 2)
TRACE t21: | | | => 7
TRACE t20: | | => 7
TRACE t19: | => 7
TRACE t18: => 7

=> 7

Example 2

remote> (demo/bar-ex 5)

output:

TRACE t22: (demo/bar-ex 5)
TRACE t23: | (demo/foo-ex 5)
TRACE t24: | | (core// 5 0)
TRACE t24: | | => com.github.jlangch.venice.VncException: / by zero
TRACE t23: | => com.github.jlangch.venice.VncException: / by zero
TRACE t22: => com.github.jlangch.venice.VncException: / by zero

Example 3

remote> (demo/factorial 5)

output:

TRACE t25: (demo/factorial 5)
TRACE t26: | (demo/factorial 4)
TRACE t27: | | (demo/factorial 3)
TRACE t28: | | | (demo/factorial 2)
TRACE t29: | | | | (demo/factorial 1)
TRACE t30: | | | | | (demo/factorial 0)
TRACE t30: | | | | | => 1
TRACE t29: | | | | => 1
TRACE t28: | | | => 2
TRACE t27: | | => 6
TRACE t26: | => 24
TRACE t25: => 120

v) Turn off tracing

Caution

Do NOT forget to turn off tracing, unless you shutdown the remote application anyway.

remote> (trace/untrace-all)

   

Hybrid Encryption

The hybrid encryption model takes the symmetric encryption to the next level by using asymmetric encryption to securely establish symmetric keys between the client and server REPL. Thus providing authentication between the two REPLs that exchange keys.

Hybrid Encryption protects against Man-in-the-Middle attacks and provides a TLS, SSH grade security level.

   

Demo

To keep the demo simple, colocated client and server REPLs are used. But the client REPL and the server application with the remote REPL can run on different hosts.

The remote REPL Hybrid Encryption demo requires the following file setup:

demo
│
├── venice-1.13.10.jar
│
├── client
│   ├── client-config.json
│   ├── client-public.pem
│   ├── client-private.pem
│   └── server-public.pem
│
└── server
    ├── remote-repl-demo.venice
    ├── server-config.json
    ├── server-public.pem
    ├── server-private.pem
    └── client-public.pem

Create the demo top-level directory first.

Download Venice and start an Ad hoc REPL in the ./demo directory to create all the required files step by step:

cd ./demo

mkdir ./client ./server

curl -O https://repo1.maven.org/maven2/com/github/jlangch/venice/1.13.0/venice-1.13.10.jar

java -jar ./venice-1.13.10.jar -repl -colors

1. Create the client REPL keys

Create the RSA asymmetric key pair for the client REPL

  • client-public.pem
  • client-private.pem

run this script in the started REPL:

(do
  (load-module :rsa)
  (rsa/save-key-pair (rsa/generate-key-pair) "./client" "client"))

2. Create the server REPL keys

Create the RSA asymmetric key pair for the server REPL

  • server-public.pem
  • server-private.pem

run this script in the started REPL:

(do
  (load-module :rsa)
  (rsa/save-key-pair (rsa/generate-key-pair) "./server" "server"))

3. Public key deployment

run this script in the started REPL:

(do
  (io/copy-file (io/file "server/server-public.pem") (io/file "client/server-public.pem"))
  (io/copy-file (io/file "client/client-public.pem") (io/file "server/client-public.pem")))

4. Create the REPL configs

Client

The client requires the keys:

  • client-public.pem
  • client-private.pem
  • server-public.pem

Create client remote REPL demo config ./demo/server/client-config.json

run this script in the started REPL:

(do
  (def config (ordered-map "port"                  33334
                           "host"                  "localhost"
                           "password"              "123"
                           "signKeyExchange"       true
                           "clientPublicKeyFile"  "./client/client-public.pem"
                           "clientPrivateKeyFile" "./client/client-private.pem"
                           "serverPublicKeyFile"  "./server/server-public.pem"))
  
  (->> (json/write-str config)
       (json/pretty-print)
       (io/spit (io/file "./client/client-config.json"))))

Server

The server requires the keys:

  • server-public.pem
  • server-private.pem
  • client-public.pem

Create server remote REPL demo config ./demo/server/server-config.json

run this script in the started REPL:

(do
  (def config (ordered-map "port"                  33334
                           "password"              "123"
                           "encrypt"               true
                           "compress"              true
                           "sessionTimeoutMinutes" 30
                           "signKeyExchange"       true
                           "serverPublicKeyFile"   "./server/server-public.pem"
                           "serverPrivateKeyFile"  "./server/server-private.pem"
                           "clientPublicKeyFile"   "./client/client-public.pem"))
  
  (->> (json/write-str config)
       (json/pretty-print)
       (io/spit (io/file "./server/server-config.json"))))

4. Start the server application

i) Create the remote REPL demo application ./demo/server/remote-repl-demo.venice:

run this script in the started REPL:

(do
  (def app """
           (do
             (ns demo)

             (def- stop? (atom false))

             (defn stop [] 
               (reset! stop? true)
               (println "Stopping demo server..."))

             (defn sum [x y] 
               (println "Function 'sum' called")
               (+ x y))

             (println "Started demo server")

             ;; just sleep until we get stopped
             (while (not @stop?)
               (sleep 1000))

             (println "Stopped demo server"))
           """)
  
  (io/spit (io/file "./server/remote-repl-demo.venice") app))

ii) Start the remote application with the remote REPL enabled in a new terminal

cd ./demo

java -jar ./venice-1.13.10.jar \
     -file ./server/remote-repl-demo.venice \
     -repl-server-config ./server/server-config.json

5. Start the client REPL

cd ./demo

java -jar ./venice-1.13.10.jar -repl

6. Access the server REPL

In the client REPL connect to the remote server REPL:

venice> !remote ./client/client-config.json

remote>

check the remote REPL Venice version

remote> (version)
=> 1.13.0

call a remote function

remote> (demo/sum 3 4)
Function 'sum' called
=> 7

switch back to the local REPL environment

remote> !local

venice>

7. Protocol Hybrid Encryption Demo

Standard out/err text produced by the remote functions are routed back to the REPL client and displayed in the according REPL colors.

REPL (client) Remote Application