Two weeks ago, I challenged myself to make a game that features pattern recognition and numbers. After a week of thinking up of ideas, coding, public testing, and tweaking, here’s the result:
Also ported it to Android using Intel XDK and Cordova.
Category: Software Development
Why Programming Competitions Could Be The Future (But Probably Won’t)
(Disclaimer: This post features somewhat stronger opinions that the usual so I have to be clear that while I manage the programming competitions in DevCon, the sentiments in this post do not represent the organization’s.)
Algorithm/Programming Competitions: the other type of “Hackathon”.
When you see the word “hackathon” nowadays, it refers to the one where a bunch of people are gathered into a single venue and form groups to work for a couple of days on ideas that they pitched at the beginning and end with a final pitch to see how good their prototype and idea is. Best pitch wins.
On the opposite end there’s the original OpenBSD type of hackathon. People are still gathered in a single venue, but there is no pitching – just coding. It can be a directed coding venture, like in the first hackathon where they worked on some parts of the OS. It can also be a creative, free-form one, like Global Game Jam where game developers make any game for 48 hours as long as it’s somehow related to the theme.
There can also be prizes in the latter, but it’s mostly a fun exercise for developers. And there’s no business stuff and pitching to worry about.
Programming Competitions have nothing to do with these two. It’s only because hackathons have become somewhat famous spectacles recently that people think that “get programmers in a single venue and make them code” automatically makes it a “hackathon”.
A programming competition is a race. First team to make programs that can solve the contest problems wins – it’s as simple as that. No pitching. Strict rules. And a lot shorter than most hackathons.
—
I believe that Programming Competitions have the most potential out of these three programming events…
Programming competitions are an objective test of programming skill. If your program doesn’t solve the given problem, you won’t get a point for it.
Compare this with college. As weird as it may sound, you can graduate with an IT-related degree without knowing how to program, whether through deception or through plain incompetence of the school. And this happens more than we’d like in the industry.
Let’s imagine programming competitions somehow become famous throughout the country in a few years time. Not basketball-level fame, but maybe soccer-level. Say there’s a persistent online Philippines-only HackerRank-like competition where anyone from the top-tier universities down to the remote colleges can test their skills. Assume that the anti-cheating and user-verification measures are good enough that graduates can put their “programmer level” on their resumes and that employers will honor it.
Imagine what would happen if students are confident of their programming skills that some of them even show them off. Imagine its impact in the industry and the academe.
It’s going to be a crazy world, won’t it?
Certainly a bigger impact than what the other hackathons can do.
—
…but given what we have right now, there’s very little chance that potential will be reached.
The opportunity is there, but there are hurdles that may be too much for anyone interested in taking it.
First problem: no one cares.
Just a quick background: I’m not a competitive programmer. I solved programming problems (Project Euler, topcoder archives) back in college just for fun. I never practiced for or competed in any live event other than the odd Google Code Jam where I never intended to pass multiple rounds.
But I understand the importance of testing students for their programming skill. After the DevCon Code Challenge Cup, I decided to code a quick online judge in Rails and in mid-2013 started to hold mini-programming competitions in various schools around the country. You can try out a demo here.
And after 2 years of doing this, I’ve had a couple of striking revelations.
One of those was how little people cared about programming. Sure, the people who invited us were really grateful to have the opportunity to test the programming skills of their students, but very few of the organizers were curious about the problems given to the contestants. I would understand the hands-off approach if they were school administrators, but many of those who invited us were IT students and teachers.
None of them reviewed the problems before I gave them to the contestants, checking if the problems were too easy or too hard or simply out of scope of their curriculum. Only a couple of them asked me about the problems after the contest even though that would have been the best time to ask for advice to improve their programming skills.
And there you go, the reason why we’re churning out weak IT graduates: the people in position don’t care about things that they should be passionate about.
The previous problem is compounded by the problem at the other end: a lack of understanding of the relative skill levels of programmers.
Programming isn’t a sport, it’s a skill and a craft. It’s not easy to gauge programmers and we are left to use vague terms like “beginner”, “skilled”, and “expert”.
And anyone who conducts a programming competition to non-competitive programmers will see that “beginner” is not adequate for describing college students. They’re all beginners, but the difference between their skill levels can be great.
Yes, that’s a euphemism for “college students suck at programming”.
Go to the demo link above and read the problems (if it’s still in Practice mode, wait until the Contest mode starts). Even though the problems can be answered by anyone who has just taken CS classes on Java programming and algorithms, in our experience only students from the big 3 universities were able to consistently answer more than 7 questions within the time frame. The next tier of universities can answer around 5, while the rest even struggle with 3.
If aren’t a programmer and don’t know how easy it is to answer just 3 in that list of problems, let’s just say anyone taking a good Programming 101 class should be able to do that in 4-5 lab sessions.
So yeah, most college programming classes are horrible. If you’re going to hold an open (non-invite only, non-ACM-ICPC) programming competition today, you ought to lower the difficulty of the questions. Otherwise, many of your contestants will get very low scores.
You don’t want that to happen. Put yourself in the shoes of someone who either was just curious or was pressured to join the contest. What will they think when they leave the event getting only one or two correct? Will they look forward to the next competition? Will they help promote future events?
Maybe, but probably not.
Even now, I still occasionally get cases where contestants get depressingly low scores (especially if I’m not familiar with the school). We’re lucky that the contests are confined within the schools so we don’t need to worry about negative publicity, but this brings up another point: unless you’re like me who can conduct more than one contest a month, most programming competition organizers do not have the experience of holding contests on a regular basis to learn from their mistakes.
Combine all these three points and you’ll have an idea what will happen when someone conducts an open programming competition in the same manner as the other hackathons.
Venue, logistics, sponsorships… no problem. Then they’ll find people who had past programming competition experience to be the judges and write the problems for them. But since it’s the first time for all of them, the judges will write difficult problems and the organizers will not look at the problems.
Fast-forward to the end of the event. Many teams joined, but only a few of them were competitive programmers. One of the latter won, and the rest were left with a score of 1 or 0.
There won’t be any negative PR (it’s not like it was a bait-and-switch), but there won’t be enough positive buzz either to make a bigger and better follow-up event. Instead of spreading the word and encouraging more people to join in the future, many will just go home and say “eh, at least I tried” and leave it at that.
On to the final point.
When you’re in my shoes as someone who has programmed over half of my life (a good chunk of that professionally) and as someone who recently has conducted programming competitions, you’ll come to hate programming competition-level problems.
I hold programming competitions in schools to show students that they need to understand their programming skill level and work on improving that to the point that they are comfortable with it even under pressure. This is the skill that I want to cultivate into students, and it will do well for them as they enter work.
But there comes a point where the problems are just too tricky that it’s no longer a test of programming skill, but a test of knowing trivia from Computer Science and Math that you will almost never get to use in the industry.
Simply put, high level programming competition skill does not equate to high level software development skill.
High level programming competition also relies on quick hacks – computationally efficient, but ultimately ugly kludges. Good Software Engineering requires you to avoid these hacks lest they become technical debt that will cripple your system and your team in the long run.
Another thing, the deeper you are into complicated problem solving, the more prone you are to The Complicator’s Gloves.
This pitfall of forgetting the original goal of having open programming competitions can be avoided as long as your organizers and judges are developers from the industry whose years of experience have removed any romantic feelings for CS trivia in favor of more pragmatic approaches to software engineering. But then again, there are far fewer CS/SE hybrids (i.e. well versed in both) in the local scene than Developer/Designer “unicorns” so good luck finding them.
—
Long story short, Programming Competitions are one of those things that look great on paper, but immediately reveals why no one does it the moment you try to do it on your own. (see also: local Developer Bootcamps)
Can anything be done to address the problems above and push these contests forward and take Philippine IT students’ skills to the next level?
Of course. But it’s going to take a much more dedicated and passionate group of people than what we have right now.
First draft/installment of my new Ruby deployment book now out
I’ve always thought that Rails’s “Deploy” page is one of the weakest parts of the Rails website – it’s just a bunch of links with no explanation unlike, say, the Active Job guide. Online tutorials also don’t help, most of them rely only on Heroku (which recently changed the behavior of their free tier to require 6 hours of sleep per day).
Beginners are left with:
- Buy a book like Agile Web Development With Rails 4 or Rails 4 in Action which have chapters on deployment.
- Read through the docs of the links on the “Deploy” page and do everything by trial and error.
- Scour through the countless tutorials online and hope the one you’re working on is correct.
In short, newbies are pretty much screwed.
So I decided to give my own take on the “Ruby deployment for Dummies”. Unlike your usual tutorial, I wouldn’t just focus on the “How to” but also the “Why”. I’ll also look at the various alternatives for the different components of your Ruby server.
And here’s the first draft/installment: A Beginner’s Guide to Deploying Rails.
It’s only the first part so it’s free to read and download. I expect to finish this book by late August.
Please buy the book so I can get out of mini-writer’s blocks and push it out earlier. LOL
RailsFTW v0.19 now out, with Rails 4.2.0 and RubyGems SSL fix
After waking up at 4AM to the n-th “How do I fix the SSL problem in RailsInstaller/RubyInstaller?” in both project’s mailing lists, I stopped watching AGDQ for a couple of hours to spin up new Windows 7 virtual machines to upgrade RailsFTW to deal with those problems.
Version 0.19 is now out, with Rails 4.2.0 and RubyGems SSL fix.
Notes on Setting up a Rails Server in 2015 (Ubuntu 14.04, Nginx, Passenger, Ruby 2.1, PostgreSQL)
I spent a decent amount of time last month playing around with scripts for setting up a Rails server from scratch. Turns out that the whole process is simpler than what I had to go through 7 years ago when I was learning how to install LAMP from scratch.
Here’s the basic set of commands to install Nginx + Passenger + Ruby 2.1 + PostgreSQL on an Ubuntu 14.04 Server:
$ sudo su
# apt-add-repository ppa:brightbox/ruby-ng
# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7
# vim /etc/apt/sources.list.d/passenger.list
# apt-get update
# apt-get install git-core build-essential ruby2.1 ruby2.1-dev \
libruby2.1 nodejs nginx-extras passenger postgresql libpq-dev
# vim /etc/nginx/nginx.conf
# adduser deploy --shell /bin/bash --disabled-password
# sudo -u postgres createuser --superuser deploy
# sudo -u postgres createdb deploy
# gem2.1 install bundler --no-ri --no-rdoc
($ prompt means user, # means root)
There are a bunch of other stuff that you need to setup like ssh/sshd/firewall and mail/postfix settings but these commands pretty much covers all the software you need to install before you setup a production app. For example, here’s the rest of the commands for setting up Redmine:
# su -l deploy $ wget http://www.redmine.org/releases/redmine-2.6.0.tar.gz $ tar -xf redmine-2.6.0.tar.gz $ cd redmine-2.6.0/ $ vim config/database.yml $ createdb redmine $ bundle install --binstubs bin --path bundle --without development test rmagick $ bin/rake generate_secret_token $ RAILS_ENV=production bin/rake db:migrate $ RAILS_ENV=production bin/rake redmine:load_default_data $ exit # vim /etc/nginx/sites-available/default # service nginx restart
Other tutorials already cover most of the commands above. Instead of repeating them, I’ll just be going through some quick notes about the process, explaining my approach and mentioning some gotchas.
Why install Passenger and Ruby from apt repositories?
It’s much faster than installing from source.
It’s also easier to update via apt-get upgrade.
Why Passenger instead of Nginx + Unicorn/Puma/etc?
Aside from the reasons above, it requires less configuration and it’s good enough for most cases.
The only way I’d consider other setups is if you need rolling restarts or more control over your memory usage.
Why use BrightBox’s repo instead of RVM/rbenv or install from source?
Most production servers only need one version of Ruby installed so I don’t see a reason for using version managers. They might make sense in staging servers for testing multiple versions, but not in prod.
Gem installation isn’t a problem since you’re going to use Bundler anyway i.e. after installing and having bundle available system-wide, you can have it install gems locally via --path like in the example above.
So that leaves us with manually installing from source. However, as mentioned above, installing via apt is just more convenient. The BrightBox packages may sometimes not be up-to-date (DevCon is running in preview 2.2 right now), but they tend to update a few weeks after a major release.
Bonus: Why are we using an older version of NodeJS?
We’re not using a more up-to-date apt repository for NodeJS because we’re only using it for asset precompilation.
There’s a gotcha related to this that we’ll talk about later.
Security: why allow root login via ssh?
Just like the “use rvm in prod” advice, it feels like a lot of people go by “PermitRootLogin no” when my script instead uses “PermitRootLogin without-password“.
For newbies, this makes sense: preventing people from logging in as root is a great way to prevent hacking attempts. In reality, we’re not totally preventing remote “root” log-ins – we’re prevent logging in as root because we want our SysAds to log in as a sudo user so all of their actions can be logged.
For simpler systems, it may be better to just allow remote root login as long as public keys are used, hence “PermitRootLogin without-password“.
Why is the deploy user not a sudo user?
It’s not really for security reasons. Yes, having a basic pasword-less user does make our apps secure, considering that Passenger automatically switches the user to the owner of the app, but my main reason for not making deploy a sudo user is because it doesn’t need it.
It doesn’t need sudo to install gems, bundle install --path installs to a local folder.
It doesn’t need sudo to restart the app server, touch tmp/restart.txt does that.
It’s just a simple constraint I added to make sure I’m isolating the deploy user to deployment tasks.
And besides, some of you might notice how insecure deploy is when it comes to…
Why is the deploy user a superuser for PostgreSQL? Why use Ident/Peer auth instead of a password?
Same reason as root login: personal server, no need for extra security than needed. If the attacker somehow logs in as deploy, the DB is screwed either way.
In other words superuser + peer auth is about as secure as the usual password approach while also easier to setup (by just a few commands LOL).
Config files
There’s nothing special about most of the config files edited via vim in the scripts above. The apt source and default nginx site config file would just contain the recommended settings. The database config can just contain three lines: the environment (production:), the adapter (adapter: postgresql), and the database name.
As for the main nginx config file, apart from being edited as recommended by the docs, there’s also one big gotcha that will prevent your app from running in Passenger:
execjs gotcha
If your app is using sprockets and you’re using an external JS runtime like NodeJS, your app may not run because Passenger can’t find a valid JS runtime via execjs.
Reason? Execjs looks for NodeJS with the PATH, and environment variables like PATH are not automatically used by Passenger.
While there are a couple of ways to solve this, my current preferred method is to use env PATH;.
—
There you have it, my quick guide to Rails server setup.
I could write a part 2 about deployment, but I’m afraid the tl;dr to that would be “just use Mina“.