Leftovers: OSS and Sharing
For the past year, we've taken note of the many projects that the Apache Software Foundation has been elevating to Top-Level Status. The organization incubates more than 350 open source projects and initiatives, and has squarely turned its focus to Big Data and developer-focused tools in recent months. As Apache moves Big Data projects to Top-Level Status, they gain valuable community support.
Only days ago, the foundation announced that Apache Kudu has graduated from the Apache Incubator to become a Top-Level Project (TLP). Kudu is an open source columnar storage engine built for the Apache Hadoop ecosystem designed to enable flexible, high-performance analytic pipelines. And now, Apache Twill has graduated as well. Twill is an abstraction over Apache Hadoop YARN that reduces the complexity of developing distributed Hadoop applications, allowing developers to focus more on their application logic.
Apache Spark, the in-memory processing system that's fast become a centerpiece of modern big data frameworks, has officially released its long-awaited version 2.0.
Aside from some major usability and performance improvements, Spark 2.0's mission is to become a total solution for streaming and real-time data. This comes as a number of other projects -- including others from the Apache Foundation -- provide their own ways to boost real-time and in-memory processing.
The early architecture of Uber consisted of a monolithic backend application written in Python that used Postgres for data persistence. Since that time, the architecture of Uber has changed significantly, to a model of microservices and new data platforms. Specifically, in many of the cases where we previously used Postgres, we now use Schemaless, a novel database sharding layer built on top of MySQL. In this article, we’ll explore some of the drawbacks we found with Postgres and explain the decision to build Schemaless and other backend services on top of MySQL.
GNU Hyperbole (pronounced Ga-new Hi-per-bo-lee), or just Hyperbole, is an amazing programmable hypertextual information management system implemented as a GNU Emacs package. This is the first public release in 2016. Hyperbole has been greatly expanded and modernized for use with the latest Emacs 25 releases; it supports GNU Emacs 24.4 or above. It contains an extensive set of improvements that can greatly boost your day-to-day productivity with Emacs and your ability to manage information stored across many different machines on the internet. People who get used to Hyperbole find it helps them so much that they prefer never to use Emacs without it.
The Belgium government wants to reuse ‘Belgian Mobile ID’ a smartphone app for electronic identification, developed by banks and telecom providers in the country. The eID app could be used for eGovernment services, and the federal IT service agency, Fedict, is working on the app’s integration.
Communities around the world are familiar with the devastation brought on by floods and droughts. Scientists are concerned that, in light of global climate change, these events will only become more frequent and intense. Water variability, at its worst, can threaten the lives and well-beings of countless people. Sadly, humans cannot control the weather to protect themselves. But according to Silja Hund, a researcher at the University of British Columbia, communities can build resilience to water resource stress.
Hund studies the occurrence and behavior of water. In particular, she studies rivers and streams. These have features (like water volume) that can change quickly. According to Hund, it is essential for communities to understand local water systems. Knowledge of water resources is helpful in developing effective water strategies. And one of the best ways to understand dynamic water bodies like rivers is to collect lots of data.
Those are same three languages that topped RedMonk’s list in January. In fact, the entire top 10 remains the same as it was it was six months ago. Perhaps the biggest surprise in Redmonk’s list—compiling the “performance of programming languages relative to one another on GitHub and Stack Overflow”—is that there are so few surprises, at least in the top 10.
It's no surprise that C and Java share the top two spots in the IEEE Spectrum's latest Interactive Top Programming Languages survey, but R at number five? That's a surprise.
This month's raking from TIOBE put Java at number one and C at number two, while the IEEE reverses those two, and the IEEE doesn't rank assembly as a top-ten language like TIOBE does.
It's worth noting however that the IEEE's sources are extremely diverse: the index comprises search results from Google, Twitter, GitHub, StackOverflow, Reddit, Hacker News, CareerBuilder, Dice, and the institute's own eXplore Digital Library.
Even then, there are some oddities in the 48 programming environments assessed: several commenters to the index have already remarked that “Arduino” shouldn't be considered a language, because code for the teeny breadboard is written in C or C++.