Sunday 4 December 2011

What does an IT pro actually do? Bing or Google translate paid services?

S#*t! .. to begin with .. i just lost the beginning of this post which i started typing as a Facebook update LOL!

Many wonder what those nerdy guys do if everybody know how to use a computer right? Well .. hopefuly none of those guys spent X amount of years in college (or similar) just to learn how to use a computer! Damn.. losing out on those par(n)ties .. sad ... ^^

Well, i for one, i'm not the nerdy type. Actually, i'm considered by my friends, peers & family as the complete opposite. Nobody really knows what i do, but they do notice, from time to time, when after spending days literally glued to my computer screens (yes, plural^^) to what i suspect my stepfather thinks as "fooling arround with the computer" that at any given time of day i go "out" and most times don't come back 'till the next morning. Yes i like, nah, let me correct that, i loooove to party! lol.

I'm losing myself on small talk, let me get you back to what i intended this post to be: an explanation of what a guy like me does when it's glued to the computer.

This all started last thursday 1st of December 2011 when Google decided (after a long 4 months notice i must say) to deprecate it's free Google Translate Api v1 for the newly and not free v2. I received a call at 4 in the morning from my dear friend and business partner in Spain letting me know that the system was not working - not working? what do you mean? - look at it he says, no info is coming out on most of the system. Dang! - clients are already freaking out for they can't show their properties to their clients!!.

4am, a quick look at the system to find out what was wrong: the translation service was no longer translating and was returning empty strings (i.e.- nothing at all) oooops!!!!

Nothing like a good shock to wake you up! .. well, hands on, let's try to find a quick translate alternative .. nah.. nothing quick at hand.. ok, let's pay the mtf's @ google to have this sorted out in the meantime  .. setting up the billling method, pay by credit card, ok let's make it work!! .. ooops! ... can't do! must rewrite the whole code!! S#'t!!!!!!!!!!! .. half hour has gone past .. clients are still unable to fetch anything at all!! .. really? how hard can it be to "port the code" from v1 to v2 .. dig onto it a bit further .. come on.. can't be that hard .. LOL!!!!!

Another half hour gone, it's time to take action, clients can't be left hanging or else they will let me hanging and go elsewhere! Ok, take the translation scripts out, rewrite them not to take into account any translations .. PIA job, rewrite scripts on 100+ files one by one ...

Two hours later during which i seriously thought about google business strategy and how they (Google) went from beeing a cool company offering free options to the world, open source software and equal opportunity to all, to being a behemoth, a goliath in all the sense of the word, a giant oppresing the good and hard working IT people like me ^^. Two hours that i spent envying them, and, what is commonly refered in Spain as "cagandome en sus puñeteros muertos!" and the job was done, no more auto-translation but data access is ok.

Google wins. I realize they have me (us, everyone) by the balls. Their will is our command.com

Anyways, after our clients had access back to their stuff, i decided it was time to go back to bed ^^.

Friday passed by without incident and research begun ... yes.. all that time we spend on the computer is not really looking at Facebook updates you know .. research, creation, development or plain old tedious repetitive work is what takes 99% of the time we spend in front of a computer. (Ask any decent IT guy and what they'll say is: NO, i dont want to spend my free time in front of a computer)

So, after a couple of hours of looking (research) at all the useless alternatives, it all got back to the same old same old .. Google or Microsoft .. both require further reading and learning on their newly (at least for me) v2 API's (Application Programming Interfaces) in order to get back the awe inspiring feature with a set of flags that will translate whatever anyone writes into a dozen different languages effortlessly on our system.

Eat, pray, love, take a dump, and back it is ... learn the new stuff. Dig into pages and pages of documentation of both providers to only grasp the surface of what their new, more powerful, more complete API's do. And yes, also take into consideration those few courses of financing you got at school and life to figure out what's the best option. In the end for me is a mixture of both: Bing offers better pricing in bulk for large translations (more than two paragraphs that is) and Google offers better tools for quick translations (though it's more expensive for bulk).

So now that translation is no longer free, i must pre-translate all the info on the database once (through Bing service) and offer that nice real-time translation service for emails through Google's paid translation. In the end, it all means i'll have to throw a couple of houndreds more each month!! for a service that was free and through which i built upon offering my clients such remarkable features just because it was free ... Now, for me as for most programmers that built upon Google services because they were free, our clients have come to like, love and rely on such features, so .. the natural thing to do is to keep them regardless of the cost it may involve.

I tweet the following: #Googletranslate api down 4ever, time 2think about what other "free" srvcs they'll let you build upon & then start charging! p.s.- follow me @nstiac @n2works =)

No wonder Google guys get richer and richer .. any resemblance to physical(fiscal) reality is no coincidence: Give them something free until they really really rely on it, then start charging people (more) for it: highways, water, owning a car, TV in some countries, etc.. Google translate api ^^

Well that was all just thursday night and friday ... yes.. notice any gaming? any went to the movies? bowling? watched the game? .. nop.. none of that.. just facing the computer & not fooling arround on it as you can see.

If you've read all the way to here, let me get to the part where no "my son is a genious with powerpoint" kid can actually accomplish and what any decent IT pro actually does:

Wakeup saturday morning and let's get hands on to actually getting the new API's to work. A quick look at google's "copy & paste" example to realize it's a laughable attempt of example for a real-world application: dynamically append scripts to the header for each translation. Let me try to explain why this is pure bulls#!t. In real life, no one pre-thinks everything they'll say (speak) throughout the day .. so basically google's example is to pre-record everything you'll say and carry it with you on a "one-time" recording machine. If it were to be the case that throughout the day you'd need to say something you haven't recorded, you'd need to purchase another one time recorder for whatever you just thought you needed.. so.. throughout the day.. you'd end up with several houndreds of recording machines (remember they are one-time recorders so you need one for each time you forgot or actually needed a new recording)..so, regardless of the weight it would take on the last hours of the day to carry let's say 20 recording machines on you, the price tag of that would be ridiculous at the end of the month when you've bought 600 recording machines right? well.. you get the idea .. so here is where creation comes in: you need to devise (create) a nice multi-recorder that suits your needs (or in IT terms: a single script that can handle & pass queries to google's REST service without leaking memory by stacking code onto the client's header.)

It's not that it is like creting the wheel all over again, but definetely nobody needs the same wheel size, thus everybody needs a different wheel, thus i need to create my own wheel (script) .. it turns out that with a little knowledge in scripting languages it turned to be ready in a breeze (a couple of hours for the whole shebang, notice i personally needed different wheel sizes myself so i created what can be thought of an automatized wheel factory^^) .. yey!! one down, one to go.. still need to work on Bing's script, another hour or so i naively thought.. lol!!!

Turns out that Microsoft, being Microsoft, codes only on their propietary programming languages (ASP & C# to be more precise) therefore no information relating common sense and freely available scripting languages is available in their site. Furthermore, Microsoft, being Microsoft, nobody else does Microsoft, so little or no information is available elsewhere on working their API through open source multi-plattform languages. Hell!!! To make matters worse, i can not use the easypeasy widely used example for it's not meant for the pro (fee based) service which i need and which requires OAP authentication (a largely succesfull attempt at complicating things) through a mixture of time-sensitive tokens and header manipulation. Damn it Microsoft, KISS!! what about making things easier for people to use? KISS stands for Keep It Simple Stupid!

So.. give it a go.. try with simple javascript, no luck (javascript OAP implementation requires global server modifications that does not comply with anything else that a server must do but to work only with their API and leave everything else useless) .. one hour has gone by .. try with jQuery ..after much trial and error and finding their documentation is flawed (parameter clientId needs to actually be client_id) i finally got a 200 OK response .. it's empty!.. damn! .. something must be wrong .. again & again .. yup! no way of actually receiving their POST back for the mechanism used to actually get the translation result is not yet implemented in jQuery .. they do have a getJSONP but Microsoft decided to complicate things by accepting POST queries only (which no fu#[ing body does anymore) ... another hour and half gone by .. oh.. and i must let you guys know that this is all only to be able to retrieve the authorization key which i need to get every 10 minutes or so.. i haven't actually done anything to even try to get a translation back.

Ok, so i cant get the key client-coded, think.. think.. (research).. let's try to get it server-coded, send the POST directly from the server instead of sending it from the browser ... several methods come to play, most of them discarded right away due to the level of cross-scripting complication they require (use pearl script, cgi, c? .. for a simple web-based app? nah.. lets try it with PHP .. curl.. dig the manual .. (more research) .. another hour goes by, trial & error, search existing examples, nobody shares! lol .. finally got it!!!!!! yeeeeeeeyyyyy!!!!!!!! got the key.. it's not a key.. it's a largely complicated mix of urls & data. KISS!!!!!

Right, so the complicated authentication part apart, the rest should be fairly easy .. WRONG! .. i remembered again why i dont like nor use Microsoft products whenever i can .. tried client scripting without success... antoher hour & half has gone by .. allthroughout 4 or 5 hours gone with this Bing translator ultra new & better performing API!! and no results yet. No documentation, nowhere to go, asking on a forum? LOL .. best answers are ask to a supervisor whom replies 4 weeks later with.. do you still have the issue? .. k .. let's go with server-side .. shouldn't be server side .. need my clients to work on-the-fly not click-wait-get-clickagain-wait-get ... ahh!! i remembered i'm using Bing for bulk database translation not on the fly .. whooo! i'd be damned if i had to rely on Bing for that .. ok .. server sided it is ... REST is bull .. HTTP is not useful server-sided .. go with SOAP .. need to recompile the server's apache & php to enable SOAP over PHP .. done .. cool right? (well... i guess i do have some nerd on me =) ...  a couple of attempts later.. got it!!!!!

Sheiße !!! 8-9 hours to get Bing translate to work .. 1-2 hours to get Google's to work ... that should tell somebody something about both providers ..

Google script template:

<script>

//  @created on 03/Dec/2011 / time spent: 1.5hours approx.

var apiKey = "xxxxxxxxxxxxxxxxxxxxxxx";
var langSource = "es";
var langTarget = "it";
var apiurl = "
https://www.googleapis.com/language/translate/v2?key=" + apiKey + "&source=" + langSource + "&target=" + langTarget + "&q=";
var text = 'This is a test string';
  
function translate(text) {

jQuery.ajax({
    url: apiurl + encodeURIComponent(text),
    dataType: 'jsonp',
    success: function(data) {
  dosomething

     }
});

}
</script>


Bing translate PHP w/OAP access token script:

<?

// @created on 03/Dec/2011 / time spent: 9.5hours approx.
function request_new_token() {
 $obj_connection = curl_init();
 $arr_query_bits = array (
 'client_id' => 'numbersOnly!!!',
 'client_secret' => 'xxxxxxxxxxxxxxxxxxxxxxxxxx',
 'scope' => 'http://api.microsofttranslator.com',
 'grant_type' => 'client_credentials'
 );
 $str_query = http_build_query($arr_query_bits);

curl_setopt($obj_connection, CURLOPT_URL, 'https://datamarket.accesscontrol.windows.net/v2/OAuth2-13');
curl_setopt($obj_connection, CURLOPT_HEADER, 0);
// curl_setopt($obj_connection, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($obj_connection, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($obj_connection, CURLOPT_POSTFIELDS, $str_query);
curl_setopt($obj_connection, CURLOPT_POST, TRUE);
curl_setopt($obj_connection, CURLOPT_TIMEOUT, 1);

$str_response = curl_exec($obj_connection);
curl_close($obj_connection);

$matches = array();
preg_match('/\"access_token\":\"([^"]+)\"/', $str_response, $matches);
$token = $matches[1];
preg_match('/\"expires_in\":\"([\d]+)\"/', $str_response, $matches);
$expires = $matches[1];

return array($token, $expires);
 }

$token=request_new_token();


$str_auth_header = "Authorization: Bearer ". $token[0];
ini_set('user_agent', 'PHP-SOAP/' . PHP_VERSION . "\r\n" . $str_auth_header);


$arr_context = array('http' =>
  array(
      "header"  => "Authorization: Bearer " . $token[0]
  )
);

$arr_options = array (
   'soap_version' => 'SOAP_1_2',
   'encoding' => 'UTF-8',
   'exceptions' => true,
   'trace' => '1',
   'cache_wsdl' => 'WSDL_CACHE_NONE',
   'stream_context' => stream_context_create($arr_context)
);

$obj_connection = new SoapClient('http://api.microsofttranslator.com/V2/SOAP.svc', $arr_options);
$str_soap_action = 'Translate';
$arr_request_arguments = array (
   'appId' => '',
   'text' => utf8_encode('This is a test'),
   'from' => "en",
   'to' => "es"
);

$obj_soap_response = call_user_func(array($obj_connection, $str_soap_action), $arr_request_arguments);
$str_result = $obj_soap_response->TranslateResult;


echo utf8_decode($str_result);


?>


LOL!!!! 2 days work for grabs .. this is what we IT pro's do .. no gaming .. no constant facebook .. no twitter all day long .. just plain old work. cheers =)

p.s.- Ask any computer savyy kid if he really understands any of this ^^