# How to set up "All-Spark Notebook" on AWS EC2 Cloud Service

My little laptop is not really going to make it through the HW2, so I finally decided to spend some time (and potentially money) to set up a cloud server for this course. You can set up yours, too!

Here is how:

1. Go to https://www.awseducate.com/Registration, Apply for a student account at the AWS Educate program. Give them a few days. 
2. Log in with the account once you got it.
3. Go to http://console.aws.amazon.com/ec2, click on "KeyPairs" > "Import KeyPair", and upload your SSH public key. Mine is at `~/.ssh/id_rsa`. Call it `id_rsa` then -- I'm not a creative person.
4. Go to https://store.docker.com/editions/community/docker-ce-aws, click on "Docker For AWS (Stable)" > "Next".
5. Under "Parameters" > "Swarm Properties" > "Which SSH key to use?", select the `id_rsa` from the dropdown menu. Click "Next" twice, check the "Capabilities" checkbox, then click "Create".
6. You will be redirected to https://console.aws.amazon.com/cloudformation/ . Wait ~10 minutes for the Status to change to "CREATE_COMPLETE".
7. Click on "Docker" (name of your stack) > "Outputs" and click on the link following "Managers".
8. Pick one of the Managers. Click on it. 
9. Click on "Actions" > "Networking" > "Change Security Groups".
10. Check the group with "Security Group Name" of "default". Click "Assign Security Groups".
11. Click "Connect". Copy the "example". It will be a shell command that looks like `ssh -i "id_rsa.pem" root@ec2-18-219-205-39.us-east-2.compute.amazonaws.com`.
12. Change the username `root` above into `docker`. Put this in a terminal and run.
    - Upon first time of connection, you might need to type `yes` when seeing `Are you sure you want to continue connecting (yes/no)?`.
    - When you see `Welcome to Docker!`, you know you are there.
13. ~~Make sure everyone (it's actually just you there) can read, write and execute everything in the home directory: `chmod 777 ~`.~~ Make a new folder: `mkdir Jupyter && chmod 777 Jupyter`.
14. Run the following command to initialize our container: `docker run -v  $HOME/Jupyter:/home/jovyan -it -p 80:8888 --entrypoint "bash" jupyter/all-spark-notebook`.
    - `-v  $HOME/Jupyter:/home/jovyan` maps the home directory on your EC2 machine to the home directory in the container. In the container, you are Jovyan.
    - `-p 80:8888` maps the `8888` port in the container to the `80` port of the EC2 instance.
    - `--entrypoint "bash"` overrides the default behaviour of `all-spark-notebook` of starting the Jupyter server directly. This will instead give us a shell.
15. While you are waiting, let's finish setting up the HTTP access. Go to https://console.aws.amazon.com/ec2/v2/home?#SecurityGroups:sort=groupId, click on the "default" group. Click on "Actions" > "Edit inbound rules" > "Add rule", set "Type" to "HTTP", and click "Save".
16. When the Docker finishes setting up the container, your shell prompt should look like `jovyan@402896de02a1:~$`. 
17. Generate a config: `jupyter notebook --generate-config`.
18. Create a password via  `jupyter notebook password`.
19. Now you can start the Jupyter Lab server via `jupyter lab`.
20. In your web browser, go to `http://ec2-18-219-205-39.us-east-2.compute.amazonaws.com/` -- remember to replace this with your EC2 instance's public DNS address.
21. Enter the password you just created. Volia!