The Elixir of concurrency
Elixir is a fairly young language that was born when José and a few Rails developers tried to create a modern language optimized for concurrent, distributed, lightweight processes
They wanted a modern Ruby-like syntax with a well-tested process manager, the Erlang VM. The result is Elixir, defined as a dynamic, functional language designed for building scalable and maintainable applications, a correct but vague affirmation which doesn't do justice to its power and elegancy.
I recently compared the move to Elixir from Python as a similar leapfrog to moving to Python from Java. It feels like something new, modern, powerful, with killer features that you don't want to renounce to.
In Python I found a REPL, list comprehensions, a super clean syntax and decorators. Elixir brings lightweight supervised
processes, pattern matching, a fully functional programming language, pipes and a terrific build tool: mix
If you've never written functional code, the jump is significant. I took a Scala course a couple years ago and I've needed almost two full weeks to write production code in Elixir. The language is young, Stack Overflow is of no help —no kidding, that is a big deal—, and there are few libraries in Github.
A small community also comes with some upsides: people are more motivated and willing to help, centralized tools like forums and IRC channels are still manageable, and you may even suggest changes to the language for upcoming versions.
What is Elixir for?
I had a middle school teacher who said that you can't define something by stating what is't not. However, in programming, mentioning use cases which are not suitable for the language is a good way to start.
Elixir is probably not the first choice for single core software: math calculus, CPU-intensive apps or desktop applications. Since it's very high level, systems programming is also out of the picture.
Elixir is great for web applications, standalone or using the Phoenix framework —Elixir's Rails—. It really shines for building highly scalable, fault-tolerant network applications, like chats, telecommunications or generic web services.
Why is that? Thanks to the Erlang VM, processes are really tiny, each one is garbage collected with a low latency,
they communicate by sending location-independent messages over the network using the VMs
(you can run result = Machine2.Module.function(params) on Machine1),
and spawning and managing these processes is effortless thanks to some of its abstractions.
Finally, Elixir's basic modules also shine: Plug and Router for managing HTTP requests, Ecto for relational databases and
ETS and Mnesia for distributed in-memory databases.
Many recommend Elixir if only for Phoenix, but I found that for most backend applications it is enough to use Plug and Router.
Phoenix is impressive but I believe it's a mistake to jump right into it without trying the base modules first, so my recommendation for
beginners is to hold on Phoenix until you really need it.
Elixir's novelty, the pipe operator, is a fantastic approach to working with state in a functional manner. Instead of running
readlines(fopen(user_input(), "r")).uppercase().split(), try the more readable user_input |> fopen("r") |> readlines |> uppercase |> split.
It is a language which was clearly designed to stand on the shoulders of giants, while providing modern capabilities for developers.
Elixir's abstractions
To store centralized <key, value>-like data, instead of a Singleton,
Elixir's provides an Agent. It keeps state in memory and many processes can access and modify it without concurrency issues.
The language can spawn processes much like threads, using spawn_link, but you probably don't want to do that. You'd
rather use a Task, which is basically async/await, or a Gen(eric)Server, a very cool abstraction that receives requests
from other processes, spawns helper mini-servers and processes the results in parallel, for free.
All tasks can be controlled using the Supervisor, which holds other abstractions as its "children" and automatically restarts
them when they crash.
Finally, your code is contained inside a single project which can manage different apps, with modules that hold functions.
No packages, no classes, no objects. Modules, functions, structs and basic data types.
Dependency management is
straightforward thanks to mix; builds and testing are handled by mix too. As opposed to other multi-tools like gradle, this
one is really fast.
Is that too much to process? I felt that at first, too. Give it some time and your brain will eventually think in terms of
Supervisors which manage GenServers which spawn Agents and Tasks when needed.
Let it crash
Elixir's mantra is to let processes crash. I found it shocking and counter-intuitive, but with some explanation it makes a lot of sense.
Neither developers want their code to crash nor Elixir promotes writing bad code. However, let's agree that there are many reasons besides bad programming which can make a software crash. If we have a server which runs stuff and at some point we have, say, 100 connections every second, one might crash eventually because of a bug in any component, hardware issues, a cosmic ray, or Murphy's law.
The question is: in the event of an unfortunate, unavoidable crash, how will your system react?
- Bring everything down?
- Try to capture the error and recover?
- Kill the crashed process and launch another one in its place?
For example, C uses approach 1. Most modern languages with Exceptions like Java and Python use 2. Elixir uses 3. This is not suitable for all environments, but it is perfect for those use cases which fit Elixir: concurrent network processes.
With Elixir, a single failure never brings the system down. What's more, it automatically restarts the crashed process, so the client can instantly retry and, unless there is a reproducible bug in your code, the fresh process will finish without an issue.
The bottom line is: a single client may be unlucky and crash at some point, but the rest of the system will never notice.
How to start?
Let's get our hands dirty. After reading many sites, watching hours of video and following a dozen tutorials, here are the resources I found the most valuable. I'd suggest following this order.
Getting started
- Madrid Elixir Meetup 2016-03. If you understand Spanish, this is the best intro to Elixir. Otherwise, watch All aboard the Elixir Express! which is a bit outdated but very comprehensive.
- Official "Getting Started" guide. It's the best and the most current. Follow it from start to finish, including the advanced chapters.
- Elixir School. A nice complement to the official guide. Most things are very similar, but the different approach on OTP will help you understand it better.
- Understanding Elixir's GenServer and Elixir's supervisors, a conceptual understanding are two short reads with yet another explanation of OTP features.
- Elixir Cheat Sheet. The best one out there
First projects
- vim-elixir-ide. Elixir support for
vim, not the best plugin but suitable for beginners. - Elixir examples. The Elixir guide covers all these, but it's handy to have common idioms on a single page: "string to list", "concatenate list", "optional function parameters", etc.
- Portal Game by José Valim. A complement to the sample project on the official guide.
- Elixir Koans and Exercism are mini exercises that you can use to improve your Elixir agility. On the same line, Elixir Golf proposes weekly puzzles to solve.
- Learning Elixir. Joseph Kain has a ton of content with mini projects and examples you can follow. Top quality.
- Excasts and Elixir sips have short screencasts that you can check out for reference
- ElixirConf videos contain very interesting talks which may be overwhelming for beginners, but are worth a look later on.
- Install Elixir and Phoenix on OSX. If you want to use Phoenix on OSX, you may need this help
- Phoenix Official Guide. Phoenix isn't necessary for simple web services, you can use
Plug. But for large projects you'll need a framework. Nothing like the official guide.
Getting help
- Awesome Elixir. A list of Elixir resources, where I found many of these.
- Elixir Tip and Elixir Status regularly link to Elixir-related articles and videos, and Plataformatec Elixir posts is where the language authors share news and tips.
- If you have questions about code, try the Elixir forum first, the IRC channel or Slack. The developers would like to transition all help requests out of the Mailing list, which you can use for language-related discussions.
- /r/elixir if you're into Reddit
Closing thoughts
I think that's all for the moment. I hope this post can help some beginners to get their hands on the language and start writing production code as soon as possible.
For anyone who wants to know what's all the Elixir fuss about, it's difficult to explain, especially for somebody like me who has been programming in imperative languages all his life.
When I recommended Elixir to a friend, he replied, "A highly concurrent, functional language using the Erlang VM? Don't you have something more exotic?". That's right. Elixir is exotic and use-case specific.
Unlike Python, which is my favorite imperative language and ecosystem, I can't recommend Elixir for everyone. Not everybody can spare a couple weeks to get started. Many libraries for common use cases are missing: there is nothing equivalent to Numpy or Matplotlib, and modern applications are built on top of dozens of libs, not everyone has the time or will to write library code. Fortunately, at Paradoxa I am my own boss and I make the tech decisions :)
For hackers or tinkerers it's definitely worth a look, it "won't change your perspective" like Lisp, but it will make you see that writing concurrent code doesn't need to be difficult, and that better tooling is definitely possible.
I bet Elixir will be the foundation of most devops stacks in a few years, when developers realize that the future's bottleneck won't be the CPU, but rather the number of concurrent processes and connections your backend can manage. With Elixir you only need to boot another machine in your network and let the exotic Erlang VM handle the rest.
Tags: programming, learning
Startup school
The amazing How to start a startup class just finished today.
How fortunate we are to live in a world where the most successful minds of our time can teach the whole planet through the Internet, and decide to allow that possibility.
Open education isn't just a fad, it's been a fight for a long time, and now it's starting to root strongly. Thanks to all who have joined on this fight, the world is becoming a better place.
If you're interested in education as a source of progress, don't miss the XPRIZE episode of the Tim Ferriss podcast, definitely my favorite podcast now
Learn new stuff, online, for free
These last months I've been quite busy learning new stuff. Researchers always need to keep an eye on the state of the art by reading papers, but multi-disciplinar sciences usually require a strong background in two or more areas, like biology, chemistry, math, physics or the always present computer science.
While we can appreciate a good book, it is not the ideal way to learn new skills. Books don't force you to follow a schedule, do the assignments, and obviously don't evaluate you. We all have work to do and, with no deadlines, we never invest enough time on learning new tools in depth. Most books and tutorials are abandoned at chapter 4.
University-grade learning
If you want to follow an university-like learning methodology, you will need to invest university-like time and money, enrolling in short courses or traditional Masters degrees. They are fine, but cost a lot of time and money. What happens if you just want to learn one specific tool or technique and not a whole curriculum?
Many universities offer workshops and non-official courses, typically for 3-5 days, which don't cost much money, but they are usually full-time and not really compatible with a daily job, unless you get permission from your supervisor.
In 2011, and especially in 2012, a new learning tool appeared on the internet: the MOOCs, or Massive Open Online Courses. Their format varies, but most of them cover one or half a university class (7-15 weeks) in depth, with video lectures, assignments, deadlines, and real interaction with other students, TAs and professors.
In learning, it isn't worth to to cheat at solitaire. A hard deadline is the only way to assure we won't lag behind the class, and while nobody will be watching whether we finish the course or drop it, it is up to us to decide if we want to take that opportunity to learn or not.
Resources to learn online
If you need some pointers I'd recommend to try Coursera first, since it has the best methodology, for my taste, and the one which resembles a college the most. If I'm not mistaken, it's also the website with the most number of courses. A good alternative is Udacity, which doesn't have as many advanced topics as Coursera, but it's great for beginners. Finally, the last generalistic site is Khan Academy, but unfortunately, it doesn't provide the same assignments-and-deadline mechanism that the former have. It basically contains video lectures. As a non-website, we can't forget iTunes U, Apple's video lectures directory, with classes directly recorded from campus.
Other self-learning resources, while not strictly MOOCs, are Codeschool and Codeacademy for programming, and Memrise for languages. The number of education websites is growing, so make a quick search on Google to check if there is anything else you could be interested in.
Effort, pace and cost
MOOCs surprisingly compare to real online universities in terms of quality and material level, however, real interaction with the professor is difficult and only through public forums. The number of makes up for that, as usually there is somebody which can answer you. It is really a collaborative effort.
Besides the obvious "knowledge for all" motto, there's another big advantage for me, as you can study at your own pace. Good MOOCs have precise deadlines which allow you some margin if you have a hard week at work or home, but they are not so far enough as to let you lag behind.
Now, the question is, who pays for that? Posting video lectures online has a marginal cost, but what about the TAs and professor's time? Most of the assignments are corrected by a computer or other students, but the organizers have to invest a significant amount of money.
I don't have the answer, but probably, since those courses are sponsored by a university, it probably fits into their social community service. Some professors advertise their books on the subject, which is perfectly reasonable, and finally, there is usually some self interest and promotion goal.
Whoever pays for the courses with their time and money, thank you.
Go check these links out, especially Coursera, and see if there's anything that catches your eye. Be brave, enroll, and prepare for a university-like effort. You'll learn university-like knowledge, for free.
Tags: learning