Non Rigid Face Tracking

Here's the result of applying Non Rigid Face Tracking on Brad, Rorie, and Vinny, while they were talking about Dark Souls 2.

Basically this computer vision algorithm goes like this:

  • Apply Active Shape Model.
    • Basically from a set of training images, in this case more than 1000 different images from MUCT database , we learn the mathematical model that approximately describes the pre-annotated facial shape of human. Here's a sample of image with pre-annotated points (also called as landmarks):

    • In this case, we learn the most general shape out of the 1000+ images, which we call the canonical shape. However since the general shape is not enough to describes large number of faces (some people has taller nose, more pronounced cheek, etc), we also retain 3 standard deviations from the canonical shape. Here's the sample of learned Active Shape Model:

  • For each pre-annotated points, we learn the correlative patch models
    • Which basically means that for each landmark point, we apply stochastic gradient descent (randomly pick an image from the image set for 1000 times), in order to find the patch that gives us the best response / result.
    • In layman terms, rather than learning the whole face, we try to find learn the general facial features for small region / patch of each landmark point. Here's the sample result of learned patch models
  • Use Viola Jones Classifier to find the approximate head position in our test data (which is the GiantBomb video), and apply the shape and patch models to fit in the landmark points to each detected faces. Here's the result of it

It's not perfect, it depends on the lighting of the scene, requires 'non difficult' head movement. Anything that depends on Machine Learning is also based on probability, but again this is not a state of the art, there are more robust algorithms out there.

If you're interested, I have the source code here: https://github.com/subokita/Sandbox/tree/master/NonRigidFaceTracking

I have no fucking idea why I decided to post this on GiantBomb though.

Start the Conversation

Non Rigid Face Tracking

Here's the result of applying Non Rigid Face Tracking on Brad, Rorie, and Vinny, while they were talking about Dark Souls 2.

Basically this computer vision algorithm goes like this:

  • Apply Active Shape Model.
    • Basically from a set of training images, in this case more than 1000 different images from MUCT database , we learn the mathematical model that approximately describes the pre-annotated facial shape of human. Here's a sample of image with pre-annotated points (also called as landmarks):

    • In this case, we learn the most general shape out of the 1000+ images, which we call the canonical shape. However since the general shape is not enough to describes large number of faces (some people has taller nose, more pronounced cheek, etc), we also retain 3 standard deviations from the canonical shape. Here's the sample of learned Active Shape Model:

  • For each pre-annotated points, we learn the correlative patch models
    • Which basically means that for each landmark point, we apply stochastic gradient descent (randomly pick an image from the image set for 1000 times), in order to find the patch that gives us the best response / result.
    • In layman terms, rather than learning the whole face, we try to find learn the general facial features for small region / patch of each landmark point. Here's the sample result of learned patch models
  • Use Viola Jones Classifier to find the approximate head position in our test data (which is the GiantBomb video), and apply the shape and patch models to fit in the landmark points to each detected faces. Here's the result of it

It's not perfect, it depends on the lighting of the scene, requires 'non difficult' head movement. Anything that depends on Machine Learning is also based on probability, but again this is not a state of the art, there are more robust algorithms out there.

If you're interested, I have the source code here: https://github.com/subokita/Sandbox/tree/master/NonRigidFaceTracking

I have no fucking idea why I decided to post this on GiantBomb though.

2 Comments

My October grab bag

Here are some items that I got / purchased this October 2013.

My mid 2009 Macbook Pro died after using it intensively for few months to train multiple number of SVMs. So I got myself an 11 inch Macbook Air.

This thing is small and light !

I also bought Kingdom Hearts 1.5 HD Remix. The artbook is bit sparse though.

My in-ear Sennheiser earphone was not functioning properly, so I finally decided to cave in and get myself a bluetooth headphone, Sony MDR-1RBT. Going wireless is liberating.

And finally I completed my Master Degree from Technical University of Munich ! My major (Computer Vision) was gruelling for me, but I learned a lot. My grade is just okay, I'm not complaining.

5 Comments

Squaring 2 digits number from the top of your head

Note: Again, this is boring nerdy trick similar to methods to check divisibility of numbers that I posted before.

Say that you want to square the number 73 and you don't have a calculator near you ? So what can you do ? Well, that might not always be the case, but you can use this as a party trick, or to check the results of your calculations, etc.

To be able to square 2 digits number from the top of your head, you need to be able to do these two things:

1. Multiply single digit number with 2 digits number in your head.

Well, this is not really a joke. Normally when we do multiplications using pen and paper, say multiplying 7 with 76, what we would do is to multiply the number from right to left side, and writing down the carry overs.

But in order for this trick to work, you need to reverse the order of your usual multiplication methods. Do it from left to right instead.

E.g. when you are doing 76 x 7, first remember that 70 x 7 = 49, then 7 x 6 = 42, then add 42 to 490, to get 532.

2. Remember the squares of all single digit numbers

e.g. 7 x 7 = 49, 4 x 4 = 16, etc

Now we can proceed to square 2 digits number by following these steps:

1. Find the nearest multiple of ten from the 2 digit number.

Say that you're squaring 73, the nearest multiple of ten is 70, which is 3 less than 73. Or say that you're squaring 49, the nearest multiple of ten is 50, 1 more than 49.

2. Add and subtract the difference between the original number and the multiple of ten to come up with 2 different numbers

In the case of 73, nearest multiple of ten is 70, which means that you should end up with 70 (i.e. 73 - 3) and 76 (i.e. 73 + 3). Or in the case of 49, then it should be 50 and 48.

3. Now multiply that 2 numbers together.

Since one of the numbers are multiple of ten, you will end up with multiplications of 2 digit with single digit numbers. E.g. 70 x 76 is essentially 7 x 76, while appending extra 0 at the end, which ends up with 5320, or 48 x 50 = 2400, etc.

4. Square the differences from step 2, and add it to final result

So while squaring 73, the difference is 3, and the result of 76 x 70 = 5320. What you should do is to square the difference (i.e. 3 x 3 = 9), and add to final result, thus yielding 5329. Which is the square of 73.

Same applies to 49 x 49, the previous result is 2400, and the difference is 1, thus, final result is 2401.

Sample exercises:

1. 64 x 64.

first step: nearest multiple of ten = 60

second step: 60 and 68 (difference = 4)

third step: 4080

final step: 4080 + 16 = 4096

2. 45 x 45.

first step: nearest multiple of ten = 40 (and 50)

second step: 40 and 50, (diff = 5)

third step: 2000

final step: 2000 + 25 = 2025

3. 81 x 81.

first step: nearest multiple of ten = 80

second step: 80 and 82, (diff = 1)

third step: 6560

final step: 6560 + 1 = 6561

With enough practice (just do it in your mind while washing dishes or ironing your clothes), you can perform this trick fast enough to impress your peers.

10 Comments

Checking divisibility of a number

The ability to check whether a number is divisible by, say 4 or 9, seems to be useless at first.

But it could be a time saver when you're trying to check the result of your calculation without resorting to a calculator. Or it could be a mental trick (or what Arthur Benjamin says as Mathemagic) which you can use it for fun while drinking some beers in the pub.

Here are some of them:

- Divisibility by 2:

Very simple, check if the last digit is even or not. Thus 928474712 is divisible by 2, while 361635 is not.

- Divisibility by 3:

Quite simple, sum up all the digits in that number, and check if it's multiples of 3.

Say you have a 7 digit number 9282648.

Sum up all the digits in that number, i.e: 9 + 2 + 8 + 2 + 6 + 4 + 8 = 39

39 is divisible by 3 (39 / 3 = 13), thus 9282648 is divisible by 3 ( 9282648 / 3 = 3094216 )

or you can go several steps further, keep summing up the digits, i.e. 3 + 9 = 12 then 1 + 2 = 3

- Divisibility by 4:

Check if the last 2 digits are divisible by 4( or if you didn't memorize multiplication table of 4, then check if you can divide the last 2 digits of the number twice, since, you know 4 = 2 x 2 ).

Example: 948137924 is divisible by 4, since 24 is 6 x 4. ( 948137924 / 4 = 237034481 )

- Divisibility by 5:

Check if the last digit is 5 or 0 (e.g. 242452525, or 80808083130 )

- Divisibility by 6:

Similar to divisibility by 3, but the original number should be even number.

E.g. 9282648 is divisible by 6 and 3, while 9282645 is divisible by 3 but not by 6.

- Divisibility by 7:

For divisibility by 7, I only know the trick that uses recursive method.

  1. Separate the digits of the number into last digit and the rest of the digits.
  2. Do this: (rest of the digits) - (2 x last digit)
  3. Check if the result is multiple of 7, if yes, then voila, else repeat from step 1

Example: 2443

  1. last digit = 3, rest of the digits = 244
  2. 244 - (2 * 3) = 238
  3. Not sure if it's divisible by 7
  4. last digit = 8, rest of the digits = 23
  5. 23 - ( 2 * 8 ) = 7
  6. Divisible by 7

- Divisibility by 8:

The last 3 digits should be divisible by 8. If it's hard to check if 3 digits number is divisible by 8, try to divide it by 2 for 3 times (since 8 = 2 x 2 x 2)

Example: 24752, last 3 digits = 752.

If it's hard to see whether 752 is divisible by 8, then keep subtracting 752 with 200, until you reached a number below 200, which is in this case 152, which is divisible by 8 (152 / 8 = 19).

Thus 24752 is divisible by 8 (24752 / 8 = 3094)

- Divisibility by 9:

Similar to divisibility by 3, but this time it will end up with multiple of 9, or if you keep doing it, it will end up with 9.

Multiple of 9 is very pretty, since

1 x 9 = 096 x 9 = 54
2 x 9 = 187 x 9 = 63
3 x 9 = 278 x 9 = 72
4 x 9 = 369 x 9 = 81
5 x 9 = 459 x10= 90

The sum of the digits of the result will always be 9.

Example: 1111111101 is divisible by 9, since 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 0 + 1 = 9. (1111111101 / 9 = 123456789)

or 7992 is divisible by 9, since 7 + 9 + 9 + 2 = 27, and 2 + 7 = 9. (7992 / 9 = 888)

There you go, I posted it after trying to figure out what's being outputted by OpenCV's HOGDescriptor's compute function, since it gives me a 1D vector of floating numbers with the size of 7 digit numbers. Then after pondering a while, I realized that the size of that 7 digit numbers is divisible by 9, thus it should be a flattened version of 9 normalized bins. You don't need to understand what I am saying though.

5 Comments

More digital drawings / paintings

This is another self indulgent post.

So I've been drawing on my iPad (1st gen) using Paper app, which I previously wrote about here: http://www.giantbomb.com/forums/off-topic/31/drawings-paintings/546791/

I think I managed to learn a lot in this past 2 months. Below are some of the results, please take note that most of the images are based on results from Google search

Elephant
Apple
Arctic Tern
Cat, again
Swan. You might have noticed that I messed up its wings
Coyote, which looks more like a wolf

Then I decided to switch from Paper app, to Procreate app, which offers layer functionality, more colors, and more brushes. Essentially it's more flexible. At that time I also decided that I should learn to draw realism drawings / paintings. If I managed to learn it, then great, else I still managed to learn something.

Water drop. You can see that I messed up the wave created from the water drop.
Candle flame. This is actually quite straightforward, since there isn't lots of things to draw
I decided to mix things up a bit. Water flame. You can see how my mind works, not stable.
Snake. The scales on the snake is really really hard to draw. I totally messed up the scales.
So I decided to draw a fluffy cat with a fluffy yarn. I like it a lot because of the pink background. Don't ask me why.
Ladybug, again another drawing based on Google Image search. I am actually trying to learn how to draw water droplet again.
I actually took a picture of a tomato, then try to draw it based on the photo. I am still trying to learn to draw water droplets.
Fluffy rain. Fuck it, I learnt bit on drawing clouds. Let's draw fluffy kittens raining from the sky.
How clouds are formed .Here you can see how my mind degenerates. You might have noticed, they are the same faucet, just scaled and translated. I also used it to do corner and feature detections while learning computer vision.
Originally I was learning how to draw tree frog. But something was missing, so I decided to add an umbrella and some raindrops.

Not sure what I am trying to say in this blog post. Try again and again, maybe ? Or maybe iPad is quite good as a drawing tablet.

Anyway I have more drawings on my tumblr page sub-okita.tumblr.com .

PS: Moderators, please remove the link if you find it's necessary to do so.

3 Comments

Drawings / Paintings

I've been busy with my graduate school work for past few weeks, and trying to fill in some free time in the morning by drawing / painting on my iPad.

I am self taught in drawing, and relatively new in painting. All of these are done on iPad using Paper app, which doesn't have layers, so I did stumble onto some difficulties, especially in drawing backgrounds. Most of the original images are actually photographs that I found on Google Image Search, but the inability to draw detailed backgrounds, made me focus on the objects themselves, thus creating a portrait like drawings.

For now, I am actually pretty happy with the result (considering it's self taught, and I am filling time while working on computer vision projects).

And here are some of the results:

25 Comments

I'll do the fact checking 04/24/2012 + Drew's North Korea trip

Most of them are taken from Wikipedia and Google. Please correct me in the comment section, if I made any mistake, or when you have better and simpler explanation or something to add (please refrain from using harsh words or insults, we're all here to learn)

04/24/2012's Bombcast

  • Arecibo message was broadcasted to outer space, some say it's to make contact, others say it's to demonstrate human capability to make contact
  • Brad was referring to Voyager Golden Record, where a phonograph record, containing sound and images portraying diversity of life on earth, is included aboard in unmanned Voyager spaceship
  • This might be Steelbook's official website. Owned by Glud & Marstrand A/S, a Denmark based company, they own IP rights to Steelbook cases. It's only steel cases for discs ?

Drew Returns from North Korea

  • Tea egg is basically boiled egg simmered in tea + chinese five spices. I've never made it, not sure about the recipe, but I've tried it before. Tastes good
  • Egg tart is a pastry filled with egg custard originated from Hong Kong. It tastes really really good. Below is the image and a video recipe of egg tart for the baking enthusiasts.
  • Apparently you take off your shoes during emergency landing before using the emergency slide. Not sure whether it's restricted to removing high-heels (since they could puncture the slide), but some websites state that you should remove shoes and excess clothing
  • Here's one of the inflight videos from Air Koryo (not the instruction guide though)
  • That british bloke is not Gary Whitta. Gary Whitta seems to be in US the entire time while Drew was in his vacation
  • I think the salad like noodles is Japchae, stir fried sweet potato noodles. Might be wrong though.
  • Kim Il-Sung is the father of Kim Jong-Il, grandfather of Kim Jong-Un. Kim Il-Sung is referred as the 'Great Leader', while Kim Jong-Il is referred as 'Supreme Leader' or 'Dear Leader'.
  • Nampho is a special city in North Korea, often used as port reception of foreign food aid assistance
  • Juche (Juche Sasang 주체사상) is apparently the thesis of Kim Il-Sung, mainly about the spirit of self reliance.
  • The hammer, sickle and brush, represents the three classes of North Korean society, which is the workers, peasants and samuwons (clerks, traders, bureaucrats, etc)
  • Pyongyang metro is touted to be the deepest subway on the world. It also acts as bomb shelter. You can see the glimpse of it from the Visual Traveling video below.
  • Below is a video clip of North Korean traffic lady in action.
  • The name Koryo (or Goryeo) is derived from Goguryeo the name of one of the three kingdoms of Korea (the other 2 were Baekje and Silla)
  • Taedonggang beer is the national beer of North Korea (?, I presume). Here's the advert which, based on Wikipedia, has only been broadcasted 3 times in total.
  • Based on Huffington Post's article here, gasoline clam uses gasoline, although it doesn't stink of gasoline in the end. Drew said that it uses rubbing alcohol. Below is the video of how they cook gasoline clam
  • I'm not sure whether it's the same in North Korea, but dog meat consumption in South Korea seems to mainly use Nureongi breed (and other breeds), which is specifically bred and raised in a dog farm. Below should be the picture of this Nureongi breed.
  • Here's the video created by Visual Traveling, which was mentioned by Drew. Great video.

I have no idea which sauce was Drew referring to, when he talked about his food adventure in China. Neither do I know about the mushroom look alike fried potato. Don't speculate.

Stare at this picture while traditional North Korean song is playing in the background

PS: If you don't have a premium membership yet, the 2 hours long interview with Drew about his trip to North Korea is both funny and fascinating.

  • I finally found out while watching Korean movie, the milk like alcohol is called Makgeolli, a korean rice wine, mixture of wheat and rice. It seems like they drink it with bowl (the one I saw was steel bowl). Not sure about the pancake though, maybe it's Bindaetteok.
9 Comments

I'll do the fact checking

Since fact checking ruins podcast, I'll ruin it for you.

  • The Raid, is an Indonesian film, directed by a Welsh director Gareth Evans. The original Indonesian title is 'Serbuan Maut' which literally means 'deadly assault'. The martial art used is called Silat (or pencak silat), pronounced as See-Lut.
  • It seems like Forcible Entry Tool is used by firemen.
  • According to wikipedia, Meconium, or early infant stool,
.... is composed of materials ingested during the time the infant spends in the uterus
  • In the last podcast, Brad mentioned about drinking alcohol / beer with straw will make you drunk faster. This is probably true. Because, by drinking with straw, we're creating vacuum, lowering pressure rapidly. With very low pressure, water and alcohol can boil within room temperature. By drinking with straw, alcohol will rapidly evaporate and rather than being absorbed in our intestine, it will be inhaled thru lung. Thus speeding up the spread of the alcohol in the blood circulation.

PS: I am from South East Asia, so it's highly probable that I know nothing about American culture or events.

PPS: Don't be discouraged by sending emails to the bombsquad, email section is one of my favorites.

19 Comments

My personal 2012 GOTY nomination list

By no way this is a comprehensive list, nor it is objective. This is just based on gut feeling, and a personal list of which games could be in my top 10 - 20 games of 2012 (plus it's restricted to PS3, the only console I own)

  • Kingdoms of Amalur Reckoning: Fun combat, but the story and dialogue are presented in such a way that, I skipped thru them all the time.
  • Asura's Wrath: Going to pick it up when the price falls.
  • Syndicate: Solid single player, really fun co-op
  • SSX: Fun trick based snowboarding game
  • Mass Effect 3: Should be a solid game
  • Street Fighter X Tekken: Probably the only fighting game I would have bought all year long. But still, online issues and DLC debacles ...
  • Journey
  • Max Payne 3: Looks technically amazing
  • Dragon's Dogma: Japanese take on open world RPG !
  • Lollipop Chainsaw: Probably mediocre gameplay but hilarious
  • Darksiders 2: Looks good
  • The Amazing Spiderman: will be released on PS3, 360, Wii, NDS, 3DS. This sounds like trouble.
  • Borderlands 2: Yeap
  • BioShock Infinite: Yeap
  • Assassin's Creed 3: Hopefully the change of setting will finally make me like AC series
  • Resident Evil 6: probably at least will be as solid as RE5, but nothing breathtaking
  • DmC: can't say that it will be good
  • Hitman Absolution: one of the few stealth games this year
  • The Last Guardian: will it ever come out this year ? If it will, probably it will rank as high as Journey
  • The Last of Us: to fill in the Naughty Dog fix this year
  • Metal Gear Rising Revengeance: Hope that it's as fun as Bayonetta, also go crazy with the story
  • Final Fantasy Versus 13: won't come out this year.
Start the Conversation