|
|
|
# Background
|
|
|
|
We have in the past used jupyterhub and we are more and more focusing on AI/Machine learning and needed some good system for utilizing this in both teaching and experiment tasks.
|
|
|
|
The covid-19 situation also increased the number of students working from home and the
|
|
|
|
|
|
|
|
|
|
|
|
# The vision
|
|
|
|
Create a load-balanced working environment for students and staff where they can run labs/experiments and the data should be saved between sessions.
|
|
|
|
Selected users should be able to share a restricted set of GPUs for running their experiments (ie machine learning/AI classes).
|
|
|
|
|
|
|
|
# The goal
|
|
|
|
Setup a jupyterhub environment with central authorization and with possibilities to assign "GPU" capabilities to selected user.
|
|
|
|
- Users should be able to save the assignment/file they are working on and it should be saved between sessions
|
|
|
|
- users should be able to use a centralized user account/passwd (ie LDAP/SAML/OAUTH)
|
|
|
|
- Should be able to select which users should have gpu access
|
|
|
|
- Teachers should be able to share notebooks
|
|
|
|
- Teachers should be able to see student notebooks
|
|
|
|
- The solution should be able to scale if when the numbers of users/ie students increase
|
|
|
|
- No user should be able to "hang" other users notebooks (ie due to programming error etc)
|
|
|
|
|
|
|
|
# How we did it
|
|
|
|
We broke down the issue is multiple steps as we will describe below
|
|
|
|
|
|
|
|
## Multi user working environment
|
|
|
|
We have been using jupyter notebookshub in the past so jupyterhub seamed like a natural fit for this task.
|
|
|
|
We then needed something that could scale and since noone of us has ever worked with swarm/conatiner mangement and felt that kubernetes was juts plain overkill for what we wanted we opted for docker swarm.
|
|
|
|
|
|
|
|
## Centralized login and gpu access restrictions
|
|
|
|
We choose between multiple options but since we have a gitlab server (that authorize the students via the central IT system/ie ldap) we resued that as a oath provider.
|
|
|
|
To differentiate between persons that sshould be able to use GPU or not we "abused" gitlabs group feature and so that if a user is part of a gitlab group he/she/it will be given different options.
|
|
|
|
|
|
|
|
|
