Skip to content

Commit 69636c5

Browse files
committed
Merge branch 'master' into release
2 parents 2891f78 + 6e382d7 commit 69636c5

29 files changed

+494
-204
lines changed

CHANGELOG.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,15 @@
11
# Changelog
22

3+
## 0.17.0
4+
5+
### Features
6+
7+
* Customizable Livy authentication methods [#662](https://github.com/jupyter-incubator/sparkmagic/pull/662). Thanks @alexismacaskilll
8+
9+
### Bug fixes
10+
11+
* Fix Dockerfile.jupyter build [#672](https://github.com/jupyter-incubator/sparkmagic/pull/672)
12+
313
## 0.16.0
414

515
### Bug fixes

Dockerfile.jupyter

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM jupyter/base-notebook:ae885c0a6226
1+
FROM jupyter/base-notebook
22

33
ARG dev_mode=false
44

README.md

Lines changed: 59 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -68,16 +68,20 @@ See the [Sending Local Data to Spark notebook](examples/Send%20local%20data%20to
6868
jupyter serverextension enable --py sparkmagic
6969

7070
## Authentication Methods
71-
72-
Sparkmagic supports:
73-
74-
* No auth
71+
Sparkmagic supports:
72+
* No auth
7573
* Basic authentication
7674
* Kerberos
7775

76+
The [Authenticator](sparkmagic/sparkmagic/auth/customauth.py) is the mechanism for authenticating to Livy. The base
77+
Authenticator used by itself supports no auth, but it can be subclassed to enable authentication via other methods.
78+
Two such examples are the [Basic](sparkmagic/sparkmagic/auth/basic.py) and [Kerberos](sparkmagic/sparkmagic/auth/kerberos.py) Authenticators.
79+
80+
### Kerberos Authenticator
81+
7882
Kerberos support is implemented via the [requests-kerberos](https://github.com/requests/requests-kerberos) package. Sparkmagic expects a kerberos ticket to be available in the system. Requests-kerberos will pick up the kerberos ticket from a cache file. For the ticket to be available, the user needs to have run [kinit](https://web.mit.edu/kerberos/krb5-1.12/doc/user/user_commands/kinit.html) to create the kerberos ticket.
7983

80-
### Kerberos Configuration
84+
#### Kerberos Configuration
8185

8286
By default the `HTTPKerberosAuth` constructor provided by the `requests-kerberos` package will use the following configuration
8387
```python
@@ -97,7 +101,56 @@ but this will not be right configuration for every context, so it is able to pas
97101
"send_cbt": true
98102
}
99103
}
100-
```
104+
```
105+
106+
### Custom Authenticators
107+
108+
You can write custom Authenticator subclasses to enable authentication via other mechanisms. All Authenticator subclasses
109+
should override the `Authenticator.__call__(request)` method that attaches HTTP Authentication to the given Request object.
110+
111+
Authenticator subclasses that add additional class attributes to be used for the authentication, such as the [Basic] (sparkmagic/sparkmagic/auth/basic.py) authenticator which adds `username` and `password` attributes, should override the `__hash__`, `__eq__`, `update_with_widget_values`, and `get_widgets` methods to work with these new attributes. This is necessary in order for the Authenticator to use these attributes in the authentication process.
112+
113+
#### Using a Custom Authenticator with Sparkmagic
114+
115+
If your repository layout is:
116+
117+
.
118+
├── LICENSE
119+
├── README.md
120+
├── customauthenticator
121+
│ ├── __init__.py
122+
│ ├── customauthenticator.py
123+
└── setup.py
124+
125+
Then to pip install from this repository, run: `pip install git+https://git_repo_url/#egg=customauthenticator`
126+
127+
After installing, you need to register the custom authenticator with Sparkmagic so it can be dynamically imported. This can be done in two different ways:
128+
1. Edit the configuration file at [`~/.sparkmagic/config.json`](config.json) with the following settings:
129+
130+
```json
131+
{
132+
"authenticators": {
133+
"Kerberos": "sparkmagic.auth.kerberos.Kerberos",
134+
"None": "sparkmagic.auth.customauth.Authenticator",
135+
"Basic_Access": "sparkmagic.auth.basic.Basic",
136+
"Custom_Auth": "customauthenticator.customauthenticator.CustomAuthenticator"
137+
}
138+
}
139+
```
140+
141+
This adds your `CustomAuthenticator` class in `customauthenticator.py` to Sparkmagic. `Custom_Auth` is the authentication type that will be displayed in the `%manage_spark` widget's Auth type dropdown as well as the Auth type passed as an argument to the -t flag in the `%spark add session` magic.
142+
143+
2. Modify the `authenticators` method in [`sparkmagic/utils/configuration.py`](sparkmagic/sparkmagic/utils/configuration.py) to return your custom authenticator:
144+
145+
```python
146+
def authenticators():
147+
return {
148+
u"Kerberos": u"sparkmagic.auth.kerberos.Kerberos",
149+
u"None": u"sparkmagic.auth.customauth.Authenticator",
150+
u"Basic_Access": u"sparkmagic.auth.basic.Basic",
151+
u"Custom_Auth": u"customauthenticator.customauthenticator.CustomAuthenticator"
152+
}
153+
```
101154

102155
## Papermill
103156

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = '0.16.0'
1+
__version__ = '0.17.0'
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = '0.16.0'
1+
__version__ = '0.17.0'

sparkmagic/example_config.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,11 @@
4141
}
4242
}
4343
},
44+
"authenticators": {
45+
"Kerberos": "sparkmagic.auth.kerberos.Kerberos",
46+
"None": "sparkmagic.auth.customauth.Authenticator",
47+
"Basic_Access": "sparkmagic.auth.basic.Basic"
48+
},
4449

4550
"wait_for_idle_timeout_seconds": 15,
4651
"livy_session_startup_timeout_seconds": 60,

sparkmagic/setup.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
'sparkmagic/controllerwidget',
55
'sparkmagic/kernels',
66
'sparkmagic/livyclientlib',
7+
'sparkmagic/auth',
78
'sparkmagic/magics',
89
'sparkmagic/kernels/pysparkkernel',
910
'sparkmagic/kernels/sparkkernel',

sparkmagic/sparkmagic/__init__.py

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,11 @@
1-
__version__ = '0.16.0'
1+
__version__ = '0.17.0'
22

33
from sparkmagic.serverextension.handlers import load_jupyter_server_extension
44

55

66
def _jupyter_server_extension_paths():
7-
return [{
8-
"module": "sparkmagic"
9-
}]
7+
return [{"module": "sparkmagic"}]
8+
109

1110
def _jupyter_nbextension_paths():
1211
return []

sparkmagic/sparkmagic/auth/__init__.py

Whitespace-only changes.

0 commit comments

Comments
 (0)