Monday, July 03, 2017

Setup MacBook almost at the speed of light

Background

I bought a new MacBook recently. It is always fascinating to setup your new machine. But it is also a pain to look for all the tools that you have on your old machine and port it to the new machine. Sometime back I started learning abount Ansible which helps to automate routine tasks. I came across a blog by Jeff Geerling who is the author of book Ansible for DevOps. Jeff and many others had used Ansible to setup their machines. I took inspiration from these guys blogs to automate the process of setting up a new MacBook Pro. Here is my experience.

Why Ansible?

Ansible is very easy to understand. It uses human readbale YAML syntax to describe the different actions which needs to be performed. Group of Ansible actions which are executed as part of a playbook are idempotent. It does not have a side effect on the setup. The same playbook can be run multiple times. Only the changes will be applied incrementally.


How did I use Ansible?

I started off by cloning the Git repository of Geerlingguy which is a good starting point. Jeff Geerling has done a very good job in terms of laying out the framework for initial set of tools. Jeff used Homebrew as package manager for installing packages. For the UI applications Homebrew Cask is used.I added some of the applications which were not exising in the original repo of Jeff Geerling.

It is very easy to get started. The repo follows the best practices from Ansible world and organizes the different topics into structure shown in th image below

Lets start with looking at some of the important files & folders from this repo.

The files directory contains additional files required for configuring specific tools. Jeff Geerling had custom options / configurations for Sublime and terminal. I added zshrc.in. The zshrc.in file is the dotfile for Oh My Zsh. We will talk about Oh My Zsh a bit later in this post.

Roles directory contains the Ansible Roles required for executing different tasks as part of the playbook. Here again, from the original repo of Jeff Geerling there were roles for managing dot files, home-brew, mas and command line tools. I added the role for managing atom packages.

Tasks folder contains list of tasks or actions which needs to performed during installation. These are organized into multiple files like ansible-setup, extra-packages-setup etc. I added a file for oh-my-zsh-setup.

The important files in the complete structure are default.config.yml and main.yml. The main.yml file is the glue that binds all the things together. This is like a main program in programming language like C# or Java. It contains references to the runtime variables, roles used and the order in which the tasks needs to be executed.
The default.config.yml file contains all the variables used by the tasks. It contains the list of tools & applications to be installed or uninstalled as part of the playbook. One of the advantage of using this approach is the applications which are installed gets moved to Applications folder as part of tasks. If we install the applications manually, sometimes we need to move them from downloads or other folder to Applications.

Apart from the applications itself, I also needed some additional libraries / tools. There were some which I was not using so I deleted those packages. Below are some of the additions / enhancements I did to meet my needs.

I made 2 major modifications to the repo of Jeff Geerling. I added the automatic configuration of Oh My Zsh and also Atom plugins & themes. Below steps were needed to make these modifications.

Setup Oh My Zsh

I like to use the Oh My Zsh as it enhances the default terminal with a better experience. It uses zsh as an alternative terminal to default terminal. Oh My ZSH is community driven framework for managing zsh configurations. It has lots of Themes & Plugins support and makes working on the terminal a really enjoyable experience. Doing a bit of Google search brought me to the GitHub repo for setting up Oh My Zsh by Remy Van Elst. I copied the zshrc.in file in the files directory. Same way I added the oh-my-zsh-setup.yml file in the tasks directory. The last step was to add an include statement to the main.yml file file to include oh-my-zsh-setup.yml file in the tasks definition.

Setup Atom plugins

Over the last few months, I had been using Atom as text editor. I used multiple Atom plugins and Themes. Atom has very good support for installing plugins using command line options. I especially like the Material UI theme which is supported by multiple editors including Atom & Sublime. I really like the minimilatic design of Atom editor.
It would be nice to have these plugins also installed as part of the machine setup. Fortunately there is an Ansible role hy Hiroaki Nakamura which allows exactly this functionality. You provide a list of Atom Themes & Plugins to this role and your machine will have all of them installed using Ansible. This is awesome. No need to go & search for plugins in the Atom UI. After the initial set of plugins, I have used the playbook for adding new ones with effortless ease.

To use the role, I added the role definition to the requirements.yml file. This file contains the list of roles whic need to be downloaded. As a pre-condition, all the roles listed here are downloaded before running any tasks. The hnakamur.atom-packages role expects a variable named atom_packages_packages. There is no difference between Themes & Plugins. I listed down all the Atom plugins & themes here. The last step was to include this role in the main.yml file.

Setup Visual Studio Code plugins & Themes

I have just started using the Visual Studio code editor. I was able to install VS Code using the default apps method from Jeff Geerling's playbook task. Similar to Atom or Sublime, VS Code has a rich support for Plugins & Themes. I found an Ansible Role by Gantsign named ansible-role-visual-studio-code. Looking at the readme file it seems to be made for Ubuntu. The role also installs VS Code editor. In my case I already have the editor installed using Homebrew Cask. I needed just the ability to install the plugins & themes.
From the code available within the repo, I found the code which is required to install the VS extension. The above role does a good job in installing VS code & extensions for multiple users of the system. Mine is a single user laptop & I did not need such functionality.

I ended up creating a file in the tasks named visual-studio-code-extensions-setup.yml. This file contains only one task of installing the extensions. The task wraps the command “code --install-extension extensionName”. The extension name is a placeholder in the above command and needs to be dynamically built. The default.config.yml defines a list of extensions in a variable named visual_studio_code_extensions. The extension name uses a specific format and it took me sometime to get hang of it. If we install the extension using VS Code IDE it works perfectly fine with just the extension name. But when we try to install the same extension using commandline, we need to prefix the publisher name. For e.g. the csharp extension is published by Microsoft. We need to provide the fully qualified name as ms-vscode.csharp.

The list of extensions can be specified as

visual_studio_code_extensions:
- steoates.autoimport
- PeterJausovec.vscode-docker
- ms-vscode.csharp

But this looks very clumsy. This is where the simplicity and flexibility of Ansible & YAML can be beneficial. We can define custom lists or dictionaries which can split the publisher & the extension name. I used this approach to define the extensions as
visual_studio_code_extensions:
- extensionName: autoimport
publisher: steoates
- extensionName: vscode-docker
publisher: PeterJausovec
- extensionName: csharp
publisher: ms-vscode

The tasks file then concatenates the publisher & the extension.

Next steps

There are still some manual steps involved in setting up the machine. As of now, I did not find a way to use Ansible to install applications from Apple App Store. I had to manually install one app Blogo which I used for writing this blog post. I am still looking for ways to automate this. There might be a way to invoke a shell command using Ansible which might allow to install App store apps. I have not tried it so far. Better way in my opinion would be to have a Ansible Role which can take a list of Apps to be installed and silently install them.

[Update]: I received a tweet from Jeff Geerling that there is mas role defined within his playbook which can be used to install apps from App Store by specifying the email & password linked to the Apple Id account. I will try this approach and update the contents accordingly.


Conclusion

At the end of the exercise, my laptop was setup and looked like below

All the applications that you see as well as additional ones which are not visible in this screen (like VS Code) were installed using Ansible playbook. It took just one single command to get these apps installed. The setup can be replicated to any other MacBook with minimal changes. Automating the installation steps have saved me much more time to do useful stuff (like writing this blog :)). You can also setup your Mac using similar steps. If you wish to do so refer to the readme file available at my Github repo. I intend to keep updating this repo with the changes that I am making to my dev environment. Feel free to contribute to this repo.

Submit Apache Spark job from IntelliJ to HDInsight cluster

Background This is 2nd part of the Step by Step guide to run Apache Spark on HDInsight cluster. In the first part we saw how to provision t...