Ever since I was in high-school I was interested in fresh ideas. Then I went to expo events in Bucharest, our “Silicon Valley”. Later I moved to Bucharest and I went to many informal meetings where many projects are presented. In most cases projects ware personal tech projects with a little business flavor.
This week I went to the Innovation Labs Demo Day. Innovation Labs functions like an accelerator for technical guys. It begins with a hackathon and ends after 3 months with a demo day.
From the very beginning, one of the core services TheWebMiner provided was aggregated data and insight into the mobile app landscape. We managed to offer our clients custom aggregated data for all major mobile app marketplaces (iOS AppStore, Google Play, Amazon AppStore etc.) as well as primary analysis on the extracted data.
Sooner we have worked at a project with restaurants and other public locations. And for this we needed an estimation of number of all the restaurants and other public locations as pubs and cafes. Continue reading →
These days I have published a Python 3 module intense used at TheWebMiner. The module it’s called PyTor and it’s available here: https://github.com/adibalcan/PyTor. It helps us to avoid CAPTCHA or other mechanisms for blocking the robots that crawl fast, websites. With this module you can detect when a website ban your IP address and you can easily change it (actually this happens automatically). Now this module it’s public and you can use it in your application.
We often run multiple task on the same server… and while ago we have used screening to manage multiple tasks/screens on server. Now we use Byobu because it’s simple to use. With Byobu the challenge is to setup function keys to work with your windows SSH client. After we try many ssh clients and many setups we discovered a perfect match between MTPuTTY ( who usees PuTTY ) and Byobu.
How to setup PuTTY to work well with Byobu:
Setup Keyboard in PuTTY config:
Terminal -> Keyboard -> The Function keys and keypad -> SELECT Xterm R6
Setup PuTTY connection:
Connection -> Data -> SET Terminal-type string AS xterm
This is all
I hope that is useful for you and you can avoid complications.
We often need to process big text files (larger than 100 mb) and we discovered that best text editor for this is Vim and gVim (windows version). Also a powerful mode to process text automatically is to use regular expressions (also called RegEx).
Using RegEx in Vim
Vim doesn’t support standard RegEx, but we built a tool that converts standard regex to Vim regex. This tool it’s available here: RegEx to Vim.
A couple of days ago I saw on the internet some photo statistics with average man and average woman from different regions around a world.
The average women face around the world
The average mens face around the world
An interesting question it’s How are made?
There are made using many layers of photos with opacity (alpha value). This means if we want to do mean of 3 photos we need to put 3 overlapping photos with 33% opacity for each. In many cases we can’t obtain a very sharp image. For example above photo results are photoshoped after overlapping.