Million dollars idea
Published 4 months ago
This is a story on how I got to know about Unity3d
Around time when I was finishing the school, I attended a course on Scientology (Applied Religious Philosophy). What was interesting is that Ron Hubbard advised to visualize all unknown phrases – to make a clay demo, draw a picture (use pictorial dictionaries), make a movie/cartoon etc…
As I was studying Chinese, this sounded double-interesting to me (some hieroglyphs look like what they represent, say, hieroglyph Sun developed from sun picture).
Anyway, some time passed, and I had an idea that it would be cool if some computer program could do that – visualize text, or, say, convert it to a movie/cartoon automatically. I went to Kyiv Institute of Cinema and Theatre, department of animation and talked with students about the idea. I said that as there is machine translation programs (services) like Google Translate, so it could be great to make a program which converts a text into a … movie … automatically.
It was a “Boom”, “Chain reaction” started. Students retold the idea to teachers. After some time Russian movie company took this idea. Bazelevs Innovations in collaboration with ABBYY Compreno got 5 000 000 dollars from Skolkovo Fund to make a program to convert text into audiovisual sequence. Technology was called “FilmLanguage”. On their site they were looking for Talented Unity3d Programmer. This is how I got interested in Unity3d. Bazelevs Innovations decided that such program works best for comedy scenarios, so they collaborated with Paramount Comedy on site (translated as “laughing out loud”) where a game using FilmLanguage was run.
Users could type Russian phrases – commands for two robots, and they would perform actions like text-to-speech synthesis or playing animations like walk, wave hand etc. When a button was pressed, a text was converted to audiovisual sequence, and then, supposedly, automatically published on Youtube (this is their channel – Chromosapiens ). Soon, this project disappeared and seems to be closed now.
But I took the idea to use Unity3d and made a small research to find out what can be done to make such program using free Unity3d assets.
To make Unity3d program for text conversion to audio-visual sequence we would require:
• Unity3d game engine, of course
• SharpNLP. It can do part-of-speech tagging and similar simple but useful tasks for English texts. SharpNLP needs some changes to work with Unity3d.
• Carnegie Mellon University Motion Capture library. It consists of over 2500 animations (many are similar) for Mecanim/humanoid rig (though I’ld expect to see these in LEGO-technic style).
• Text-to-speech library. I use this one.
• Code. Now, having all these, are you able to make a program which converts text into audio-visual sequence?

Ivan Donets
looking for Freelancer job: Junior Unity3d/C# - Programmer