Monthly Archives: May 2014

Challenging users to data science

A problem well known in the data science world is the mismatch between people who have the data and people who know how to use it. On the other hand data scientists complain about the difficulties of the scrapping process and more exact, the difficulties of obtaining the data. For this mismatch Kaggle was created, trying to mediate a connection between data and analysts.

The platform was born on this principles and creates a competition between users which must update solutions to diverse data sets and so to win points, and, in the end, money.

On the other side, the uploader of data gets a number of possible solutions of analysis to his data sets, from which he can choose the most appropriate to his interests.

A very interesting case study, and a powerful demonstration in favor of Kaggle capabilities is the collaboration that the platform has, with NASA and Royal Astronomical Society, in which the challenge was to find an algorithm for measuring the distortions in images of galaxies in order for scientists to prove the existence of dark matter. It seems that within a week from the start of the project, the accuracy of the algorithms provided by NASA, and obtained in studies started back in 1934 and continued to that time was reached. More than this , within three months from the start of the project, an algorithm was provided by a user, that was more than 300% more accurate than any of the previous versions. The whole case study can be found here.

 essentially, the fun thing about Kaggle is that the winners of the competitions are folks around the world with a knack for problem solving, and not always degrees in mathematics. And degrees don’t matter on Kaggle; all that matters is result. 

 

 

Enigma Analytics

Without any introduction we can certainly say that Enigma is a tool that should not be ignored by any data enthusiast. First introduced to the wide public at TechCrunch Disrupt NY 2013 where this start-up was the grand winner, it has gained popularity by simplicity of use and wide availability of its content.

Enigma allows its users to explore a vast amount of publicly available although not easy to obtain data. The service pulls its data from more than 100,000 data sources, a major advantage being a deceptively simple process of sifting through all the information  — a quick search for a person’s name or company brings up multiple detailed sources of information, and jumping in and playing with data is thoughtfully executed. 

By now the excellent simplistic design and usefulness of the information provided in one place has brought the company partnerships with the Harvard Business School, research firm Gerson Lehrman Group, S&P Capital IQ, and newly-minted strategic investor the New York Times.

Although by now it has proven itself a very useful tool Enigma has its ups and downs. The biggest downsize is the fact that it only has databases collected from american government and american local authorities, which is great because those datasets are public and free but they are not very useful for researchers from another countries, unless they are studying their country relations with America. Second of all, its simplistic design can be a bit confusing at first because it’s a new type of application and not all of its functions are clear. However this can be avoided if before browsing through the site you first visit the support section.

All in all , we have reached the verdict that Enigma is a great App if you are interested in public data of America, not easy to obtain otherwise.

Companies join forces against FCC

After years of pressure from ISPs, net neutrality is under threat by the FCC itself. Chair Tom Wheeler promised to revive the Open Internet Order after it saw an unceremonious defeat in January, but a leaked version of his latest proposal would let companies pay ISPs for a “fast lane” to subscribers, undermining the spirit of the original rules, which barred companies from discriminating between services. Despite Wheeler’s reassurances, this new proposal is the exact opposite of net neutrality. It could undermine both the companies of today and the startups of tomorrow. It might also be exactly the push activists need to fight back, according to The Verge .

As Washington Post suggest, more than 150 internet firms are protesting in a letter to the Federal Communications Commission. The companies asked federal regulators to reconsider a proposal that critics fear would allow Internet providers to charge for faster, better access to consumers. The list includes Amazon, Facebook, Google and Microsoft, along with dozens of other firms that called the prospect of paid fast lanes “a threat to the Internet.”

With just a week to go before the Federal Communications Commission meets to consider its proposed new rules for ISPs, the letter represents a late attempt by Silicon Valley to take a stance on the open Internet.

“Instead of permitting individualized bargaining and discrimination,” the companies wrote, “the commission’s rules should protect users and Internet companies on both fixed and mobile platforms against blocking, discrimination and paid prioritization, and should make the market for Internet services more transparent.”

The main question is whether a slow-down protest would have any impact. But it is undoubtedly worth starting a broader conversation about what the Internet community can do together to protest the FCC’s proposed rules.