Brain Dumps

First draft/installment of my new Ruby deployment book now out

I’ve always thought that Rails’s “Deploy” page is one of the weakest parts of the Rails website – it’s just a bunch of links with no explanation unlike, say, the Active Job guide. Online tutorials also don’t help, most of them rely only on Heroku (which recently changed the behavior of their free tier to require 6 hours of sleep per day).

Beginners are left with:

Buy a book like Agile Web Development With Rails 4 or Rails 4 in Action which have chapters on deployment.
Read through the docs of the links on the “Deploy” page and do everything by trial and error.
Scour through the countless tutorials online and hope the one you’re working on is correct.

In short, newbies are pretty much screwed.

So I decided to give my own take on the “Ruby deployment for Dummies”. Unlike your usual tutorial, I wouldn’t just focus on the “How to” but also the “Why”. I’ll also look at the various alternatives for the different components of your Ruby server.

And here’s the first draft/installment: A Beginner’s Guide to Deploying Rails.

It’s only the first part so it’s free to read and download. I expect to finish this book by late August.

Please buy the book so I can get out of mini-writer’s blocks and push it out earlier. LOL

My Filipino Dishes (with Science!) pt 4: Tinola

In this series of posts, I’ll be discussing the different Filipino dishes that I’ve been cooking and experimenting on for quite some time now.

« You can find out more in the introductory post.

Past dishes:
« Dish #1: Giniling
« Dish #2: Bistek

Just two quick notes for all recipes:

Most of them, when paired with rice, will serve 3-5 people.
I will only show pictures of the ingredients. I’m leaving the “take a picture of the dish” to the foodies. Go to Google Image Search if you want pictures of the dishes.

—

I hate Tinola. I hate HATE Tinola. I’ve gone through college and years of work and I’ve never ever ordered Tinola at the turo-turo and cafeteria. I’ve always wondered why people like this dish of tasteless rubber chicken and even more tasteless soup.

Then, just like Giniling, I was asked to cook Tinola for the household.

And so I went around the internet, looking for recipes and stories about cooking Tinola.

And that night I had the best Tinola in my life.

It was then that I figured out the problem with Tinola: Bistek has one subtlety (cooking time) that can make or break the dish; Tinola has a lot more than that. It was easy to see why a lot of Tinola sucked after I learned those subtleties.

—

Tinolang Manok

(Ginger Chicken Stew)

Tinola

1/2 chicken, quartered (1/4 – 1/3kg)
1 thumb ginger, chopped
1 small onion, minced
2 cloves garlic, minced
1 medium chayote, chopped into blocks
2 branches malunggay leaves
1 chicken bouillon cube
rice water from the first wash
patis (fish sauce)

Step 1: Put the chicken pieces in a pot skin side down and simmer in low heat to render the fat. You can add a bit of water to help the rendering process and prevent the meat from burning.

Step 2: After around 10 minutes, enough fat should have been rendered from the skin to allow stir-frying of the meat. Add a bit of cooking oil if there isn’t.

Add the ginger, onion, garlic, and 1-2 tbsp of patis and stir fry at medium-high heat until the chicken starts to brown.

Step 3: Add the bouillon cube and enough rice water to fully immerse the chicken and enough extra for the vegetables (ie. extra ~1cm margin). Cover and simmer at low heat for 20 minutes.

Step 4: Add the chayote blocks. Cover and simmer at low heat for another 20 minutes, stirring occasionally.

Step 5: Add the malunggay and cook for another 2 minutes (just to blanch the leaves) and serve.

—

Before I discuss the subtleties, I’d like to point out this one of those recipes where full mise en place is sub-optimal. There is enough time between steps to prepare all of the ingredients:

Peeling and mincing garlic, onion, and ginger can be done while waiting for the fat to render.
Peeling and chopping chayote can be done while the stew is simmering.
Stripping/plucking malunggay leaves from its stalks takes a crapload of time. Start doing it before you add the chayote or you might end up with overcooked chayote by the time you’re done.

—

Subtlety 1: sangkutsa

You know why you’re getting tasteless “rubber” chickens in your tinola? It’s probably because the recipe you’re using only told you to put all the ingredients in a pot and simmer for 30 mins.

This doesn’t add flavor or even texture to your chicken. Heck, you might be better off microwaving it.

The solution: sangkutsa. The word can mean “marinate”, but here we use its other meaning: stir-frying in rendered fat.

Done right, sangkutsa essentially turns our rubber chickens into skinless fried chickens. The texture is better, and the slightly browned meat has better flavor thanks to our old friend Maillard reaction. Even the browned residue will be useful later: apart from flavor, it also gives the stew its brown tint as the pot is deglazed in the long simmering period.

Subtlety 2: native chicken vs broiler chickens

Another theory why we tend to get tasteless Tinola is because the chickens themselves are tasteless. Tinola predates the modern broiler chicken and it’s safe to assume that most recipes were meant for “native” chickens. Compared to native chickens, broilers have less skin-to-meat ratio and in turn will have less fat and flavor per weight.

There are a couple of ways to get around this:

Bouillon cubes – as I mentioned before, chicken stock is rare locally so we’ll have to settle with bouillon cubes or other similar seasonings (e.g. “Magic Sarap”, MSG) to enhance the existing patis/ginger/garlic/stock flavor.
Chicken necks – they don’t have a lot of meat so people avoid them, but you’ll get extra skin and the bones give you extra flavor as stock if you use them here.

Subtlety 3: cooking time

Like Bistek, simmering Tinola for a long time improves the quality of the dish. This is normal for most types of chicken soup – long cooking time allows the connective tissue to break down and add flavor to the stock.

Even though Wikipedia notes that chicken soups take 1-3 hours to cook, 45 minutes works well for me. I’m pretty sure it’s because of the pre-cooking in sangkutsa, though if I had to cook native chicken I’ll probably extend the simmering time by 15-30 mins to get more flavor out of the well-used connective tissue.

Subltety 4+: more flavors

Apart from stock and chicken fat itself, there are still other subtleties in the flavor area.

First and foremost is the ginger. It was only when I looked for recipes that I found out that ginger is one of the key ingredients – and yet all of the Tinola I had before never even had a hint of ginger in them.

I’m not saying you should overdo the ginger and make your soup taste like salabat, but you should at least use enough to make the person eating taste and feel some of the heat from the ginger. If you can’t decide how much ginger to use, just remember that Tinola is a cold-weather dish.

Speaking of heat, most recipes call for dahon ng sili (chili leaves) to give more depth to the hot ginger stew and I would suggest beginners to use that instead of harder to prepare malunggay. I prefer the latter mainly because I can get fresh leaves from the virtually unkillable zero-maintenance malunggay trees our backyard.

On to the other vegetable aspect, most recipes also call for unripe papaya instead of chayote. They become the same texture in the end, but I personally don’t like unripe papaya in my tinola. I have a sweet tooth so I prefer chayote’s slightly sweet undertones over papaya’s bitter aftertaste. People who grew up on bitter dishes (e.g. Ilocanos with their Pakbet and Papaitan) usually prefer the former, though.

The last thing I’m going to touch is the rice water. Due to the other factors affecting the taste of the stew, I honestly can’t tell the difference between Tinola made with plain water and one made with rice water. Theoretically the rice starch should thicken the broth but I really don’t have the patience to make 2 batches at the same time and compare both. However, since I’m typically the guy washing and cooking the rice before cooking the ulam, saving the rice washings for Tinola couldn’t hurt.

—

As a side note, I never thought I’d write a 1000+ word essay on a dish I hated until last year.

—

(next recipe will be a bonus dish)

RailsFTW v0.19 now out, with Rails 4.2.0 and RubyGems SSL fix

After waking up at 4AM to the n-th “How do I fix the SSL problem in RailsInstaller/RubyInstaller?” in both project’s mailing lists, I stopped watching AGDQ for a couple of hours to spin up new Windows 7 virtual machines to upgrade RailsFTW to deal with those problems.

Version 0.19 is now out, with Rails 4.2.0 and RubyGems SSL fix.

Notes on Setting up a Rails Server in 2015 (Ubuntu 14.04, Nginx, Passenger, Ruby 2.1, PostgreSQL)

I spent a decent amount of time last month playing around with scripts for setting up a Rails server from scratch. Turns out that the whole process is simpler than what I had to go through 7 years ago when I was learning how to install LAMP from scratch.

Here’s the basic set of commands to install Nginx + Passenger + Ruby 2.1 + PostgreSQL on an Ubuntu 14.04 Server:

$ sudo su

# apt-add-repository ppa:brightbox/ruby-ng
# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7
# vim /etc/apt/sources.list.d/passenger.list
 
# apt-get update
 
# apt-get install git-core build-essential ruby2.1 ruby2.1-dev \
    libruby2.1 nodejs nginx-extras passenger postgresql libpq-dev
 
# vim /etc/nginx/nginx.conf
 
# adduser deploy --shell /bin/bash --disabled-password
# sudo -u postgres createuser --superuser deploy
# sudo -u postgres createdb deploy
 
# gem2.1 install bundler --no-ri --no-rdoc

($ prompt means user, # means root)

There are a bunch of other stuff that you need to setup like ssh/sshd/firewall and mail/postfix settings but these commands pretty much covers all the software you need to install before you setup a production app. For example, here’s the rest of the commands for setting up Redmine:

# su -l deploy

$ wget http://www.redmine.org/releases/redmine-2.6.0.tar.gz
$ tar -xf redmine-2.6.0.tar.gz
$ cd redmine-2.6.0/
$ vim config/database.yml
$ createdb redmine
$ bundle install --binstubs bin --path bundle --without development test rmagick
$ bin/rake generate_secret_token
$ RAILS_ENV=production bin/rake db:migrate
$ RAILS_ENV=production bin/rake redmine:load_default_data
$ exit

# vim /etc/nginx/sites-available/default
# service nginx restart

Other tutorials already cover most of the commands above. Instead of repeating them, I’ll just be going through some quick notes about the process, explaining my approach and mentioning some gotchas.

Why install Passenger and Ruby from apt repositories?

It’s much faster than installing from source.

It’s also easier to update via apt-get upgrade.

Why Passenger instead of Nginx + Unicorn/Puma/etc?

Aside from the reasons above, it requires less configuration and it’s good enough for most cases.

The only way I’d consider other setups is if you need rolling restarts or more control over your memory usage.

Why use BrightBox’s repo instead of RVM/rbenv or install from source?

Most production servers only need one version of Ruby installed so I don’t see a reason for using version managers. They might make sense in staging servers for testing multiple versions, but not in prod.

Gem installation isn’t a problem since you’re going to use Bundler anyway i.e. after installing and having bundle available system-wide, you can have it install gems locally via --path like in the example above.

So that leaves us with manually installing from source. However, as mentioned above, installing via apt is just more convenient. The BrightBox packages may sometimes not be up-to-date (DevCon is running in preview 2.2 right now), but they tend to update a few weeks after a major release.

Bonus: Why are we using an older version of NodeJS?

We’re not using a more up-to-date apt repository for NodeJS because we’re only using it for asset precompilation.

There’s a gotcha related to this that we’ll talk about later.

Security: why allow root login via ssh?

Just like the “use rvm in prod” advice, it feels like a lot of people go by “PermitRootLogin no” when my script instead uses “PermitRootLogin without-password“.

For newbies, this makes sense: preventing people from logging in as root is a great way to prevent hacking attempts. In reality, we’re not totally preventing remote “root” log-ins – we’re prevent logging in as root because we want our SysAds to log in as a sudo user so all of their actions can be logged.

For simpler systems, it may be better to just allow remote root login as long as public keys are used, hence “PermitRootLogin without-password“.

Why is the deploy user not a sudo user?

It’s not really for security reasons. Yes, having a basic pasword-less user does make our apps secure, considering that Passenger automatically switches the user to the owner of the app, but my main reason for not making deploy a sudo user is because it doesn’t need it.

It doesn’t need sudo to install gems, bundle install --path installs to a local folder.
It doesn’t need sudo to restart the app server, touch tmp/restart.txt does that.

It’s just a simple constraint I added to make sure I’m isolating the deploy user to deployment tasks.

And besides, some of you might notice how insecure deploy is when it comes to…

Why is the deploy user a superuser for PostgreSQL? Why use Ident/Peer auth instead of a password?

Same reason as root login: personal server, no need for extra security than needed. If the attacker somehow logs in as deploy, the DB is screwed either way.

In other words superuser + peer auth is about as secure as the usual password approach while also easier to setup (by just a few commands LOL).

Config files

There’s nothing special about most of the config files edited via vim in the scripts above. The apt source and default nginx site config file would just contain the recommended settings. The database config can just contain three lines: the environment (production:), the adapter (adapter: postgresql), and the database name.

As for the main nginx config file, apart from being edited as recommended by the docs, there’s also one big gotcha that will prevent your app from running in Passenger:

execjs gotcha

If your app is using sprockets and you’re using an external JS runtime like NodeJS, your app may not run because Passenger can’t find a valid JS runtime via execjs.

Reason? Execjs looks for NodeJS with the PATH, and environment variables like PATH are not automatically used by Passenger.

While there are a couple of ways to solve this, my current preferred method is to use env PATH;.

—

There you have it, my quick guide to Rails server setup.

I could write a part 2 about deployment, but I’m afraid the tl;dr to that would be “just use Mina“.

My Filipino Recipes (and Science!) pt 3: Bistek

In this series of posts, I’ll be discussing the different Filipino dishes that I’ve been cooking and experimenting on for quite some time now.

« You can find out more in the introductory post.

Past dishes:
« Dish #1: Giniling

Just two quick notes for all recipes:

Most of them, when paired with rice, will serve 3-5 people.
I will only show pictures of the ingredients. I’m leaving the “take a picture of the dish” to the foodies. Go to Google Image Search if you want pictures of the dishes.

—

Like Giniling, today’s recipe is so easy that a kid can make it. However, you do need to know “The One Weird Trick™” in cooking it.

—

Bistek

(Stewed Beef Strips)

Giniling

1/4 kg sirloin beef strips
1 medium white onion, cut crosswise (i.e. rings)
2 cloves garlic, chopped
calamansi
soy sauce
1/2 tsp ground pepper

Step 1: In a plastic container, marinate the beef with juice from 2-3 calamansi (use a stainer to remove the seeds), garlic, pepper, and soy sauce (1/4 cup? I just use enough to fully immerse the beef). Let the meat marinate for at least 30 minutes. I like mine marinated for more than 2 hours.

Step 2: Saute onions in oil in a pot. Again I prefer stainless steel.

Step 3: Put the meat (marinade included) in the pot along with around 1/2 cup of water and 2 tbsps soy sauce. Cover and simmer for at least 45 minutes. Stir at 10-15 minute intervals, adding a bit water when it gets too dry. Check done-ness by tearing apart a strip with a pair of forks. The meat should tear with little effort.

Season to taste before serving. We recommend adding more calamansi juice at the end for more bite. Oyster sauce can also be added to the simmering stew for the usual “body and flavor”.

As with Giniling, if you use too much soy sauce, you can try salvaging the dish with sugar.

—

Just like Giniling, Bistek is another common sight at turo-turo (eateries). It’s also called “Beef Steak” which is both wrong (when compared to the more popular American Steak) and technically right (when taking its Spanish roots) at the same time. Because of this, when you hear a Filipino mention “Pork Steak”, it’s probably not a literal pork steak, but pork chops cooked in the same fashion as Bistek

And by the same fashion, it just means “simmer marinated meat in soy sauce and calamansi juice for a long time”. For people not living in the tropics, you can try to replace calamansi with any other lime-related fruit (not lemon or orange) but I haven’t tried it myself.

The long cooking time is “The One Weird Trick™”: meat goes through the following phases when you cook it:

Soft, tender, and juicy because it’s undercooked (e.g. like a medium rare steak)

Tough and dry because it’s overcooked (e.g. overcooked steak)

Tender and juicy because the collagen has broken down (e.g. stewed/braised/slowcooked meat)

Dry and stringy (overstewed/overbraised meat)

Unfortunately, many people are not aware of the information above and think that their tough, jerky-like Bistek is a problem with prepping the meat e.g. “we should’ve tenderized it with a mallet” or “we should’ve marinated it longer”.

Repeating it again to drill it into your head, if you want tender bistek, you have to simmer the meat for a long time. The 45 minutes is enough since we’re using strips. For thicker cuts, you’ll have to wait for a future recipe that deals with that.

—

Bonus recipe:

I do this variation if I want to waste time. The only things it does to the final product are (1) make the onions look more presentable i.e. not soggy rings and (2) add a bit more flavor via the Mallard reaction.

We do this by separating the sauteed the onions and frying the meat like Tapa before stewing.

Bistek, the hard way

Same ingredients as above

Step 1: Same as step 1 above, but do not include garlic and pepper as they will burn in the searing process.

Step 2: Saute onions in oil and set aside. To save cleaning, you can do this in the pot that you would use for the next step.

Step 3: Add a bit more oil to the pot and sear the meat (set aside the marinade). You can do this strip by strip or as a whole batch. Since it’s marinated, the meat will give off a lot of liquid and the browning process isn’t as simple as a steak. You can let the juice boil off (just like frying sausages) or you can drain it into the marinade container and add a bit of oil to continue the frying process.

Step 4: Once the strips look like fried Tapa (lol), add the marinade and the rest of the ingredients (sans onions) and continue from step 3 onwards. Upon serving, top the dish with the sauteed onion rings.

—

Next recipe: Tinola »