CCS Demo Observatory
CCS Demo Observatory
Data is power, and
big data creates injustice. Organizations that control large amounts of data, for example, the entire listening or watching or reading history of hundreds of millions of people in all major countries of the world, can train algorithms and robots that drive most of the music, film, book or games sales in the world. They can make your investment into a sound recording successful or doomed. They can circumvent or help a local content regulation, reinforce, or disable a national cultural policy goal. A country may introduce national artist quotas on radio, if all the youth will be personally recommended foreign songs in their music discovery age in the very same country.
We want big data to work for small venues, independent labels, startups, great and undiscovered artists. We believe that you cannot make a successful album launch, a market entry or introduce a successful cultural policy without large amounts of well processed and correctly analysed data. We want to create a Music Observatory that integrates the small data of many small bands, small labels, small venues, small countries, and mount correct the injustice. Make algorithms transparent and the competition fair.
In 2020 most of the data is proprietary to a few, U.S.-based companies, while most of the paying audience is the European Union. This year, many European governments started to challenge the competition conditions. The tide is turning, and we do not see this as an EU-US rivalry, because many players of the American music, book, publishing, film or other creative industry share all the pains of the European creative and cultural industries.
Our Demo Music Observatory in September 2020 and got into the prestigious Yes!Delft AI Validation Lab. Our demo observatory is an example how we believe the European Data Observatory should be built. An observatory is a permanent observation point for social and economic data. We want to prove that this process can be made cost-effectively and efficiently, providing a high-quality, valuable and timely product by employing best practices in research automation and open source software, using open data in open collaboration with the music industry, artists, technicians and managers.
Currently the website which will be updated frequently after 14 September 2020. Each time a refreshment is made, the entire website in html, the downloadable data catalogue in pdf and epub ebook is re-created with new data tables, latest citation information and visualizations.
This is the twin demo observatory of music.dataobservatory.eu/, which is still only partially finished, but it is already refreshing daily and working autonomously.
A demonstration and proof of concept that a modern, European data observatory can be in large part automated, and adhere to the highest standards of statistical disclosure, reproducible research and open policy analysis (see 1.1 Evidence-based, Open Policy Analysis).
Applying the highest standards of open collaboration and open policy analysis, we make the entire source code of our data creation open source, and subject it around 10 September 2020 for peer-review as statistical software. Gradually the entire code for creating this mini-observatories indicators will be published under code.ccs.dataobservatory.eu. Our critical software components, such as regions.dataobservatory.eu for creating historical sub-national (provincial, regional) statistical comparisons, iotables.dataobservatory.eu to create gross value added, employment and tax multipliers for all EU countries, or our tools to harmonize survey data with European or global surveys is not only open-source, but either peer-reviewed, or is under review.
We are presenting some exclusive indicators that were compiled from pan-European statistical questionnaires, originally not intended for the music industry (see 220.127.116.11 Ownership of CD players and 18.104.22.168 Ownership of smartphones as examples).
We will present some indicators where we have no publishing rights in visualization, but most of the indicators will be available in automatically, daily refreshing tables, with daily refreshing bibliographical citation files in this website.
While we believe that relying on open data offers the best value for money, and should be a starting point, our 1.4 open collaboration is not based entirely on open data, on the contrary. It offers a transparent, opt-in, secure model for the industry to pool, join, integrate highly confidential, proprietary data with open data in the best possible way. We are presenting here some indicators where we have no publishing rights in combinations with open data, without revealing the data itself, to show that the benefits of data integration are enormous. (See our example about the “royalty gap”.)
In fact, our approach to provide the Demo Music Observatory, and possibly a large part of a future European Music Observatory for free relies on future collaboration that build upon an opt-in open collaboration into collaborative data integration or private data integration.
In a 1.3.3 private data integration we are offering industry, research and policy partners highly automated tools that bring into their research teams the open data, and combine it with their confidential, proprietary data. In the 7 Innovation you can read about how we want to make forecasting, AI & machine learning, royalty valuation and copyright infringement compensation calculation, or how to decrease research costs significantly by automated reporting and documentation.
Our start-up, Reprex B.V. (website: dataobservatory.eu, short introduction in the Annex) has applied to the Artificial Intelligence Validation Lab of Yes!Delft, which is considered to be one of the second best university-backed high-tech startup incubator program in the world. Our aim is to ask for their help to find a successful business model for an open source, open data, open collaboration based reproducible research company that can finance its operations mainly from exclusive data services to participants of its observatories, and to a less extent, public funding for public data infrastructure. Our approach is based on a combination of collaborative data integration and private data integration that partly finances and enables an open collaboration that create plenty of open data. Our unique value proposition is that by significantly lowering the data acquisition and data processing costs of our partners, save a large cost base for value creating activities such as 7.2 forecasting, 7.1 AI & machine learning,??royalty valuation and copyright infringement compensation calculation, 7.3 impact assessments for grants and policies, or how to decrease overhead research costs significantly by automated reporting and documentation.
The 1 Our Approach for Creating a Music Observatory details a bit more this concept. The Annex gives even more technical details, and to make it easier to understand, we are releasing about 50 indicators following the planned structure of the European Music Observatory for free. At the same time we are reaching out to potential partners on international, national, sub-national and even individual levels to join in with letters of intents to cooperate, budgetary commitments, data sharing commitments. Clicking through or reading through this document you will see initially large gaps. We published some exciting indicators, but we want to select the rest of the indicators for this Demo Observatory with our partners, prioritizing the needs of those who take our invitation first.
Our aim is to create a scientifically valid and authoritative data source that can be used, for example, as evidence in front of royalty tribunals or courts. Our datasets will not only be daily refreshed, but each change will have an authoritative copy and a digital object identifier (see for example.) Each table will receive a doi identifier, and whenever new data comes in (automatically), a new version will be sent automatically to figshare, and a new version doi will be retrieved. (We will soon contact the EU’s similar Zenodo data repository for a cooperation possibility, figshare was the first choice of our academic partner IViR.)
We believe that this could a very logical continuation of the work of CEEMID, which came to existence in less data rich countries of the EU with the same purpose in 2014. Our work (see Central European Music Industry Report 2020) was also put on stage on as a good example of evidence-based policy making on CCS Ecosystems: FLIPPING THE ODDS Conference – a two-day high-level stakeholder event jointly organized by Geothe-Institute and the DG Education, Youth, Sport and Culture of the European Commission with the Creative FLIP project. We would like to find a way into the Creative FLIP program with a grant application.
We believe that this observatory together with its twin music.dataobservatory.eu will provide a useful proof of concept for a future European Music Observatory, and will find many contributors from the earlier CEEMID partners and beyond.
Credits: Video & related static images: Line Matson; music: Moon Moon Moon; retrospectively harmonized survey data: Marta Kołczyńska; COVID indicators: Istvan Zsoldos; publication automation: Daniel Antal & Sandor Budai; written by Daniel Antal.
How can you join?
Our CCS Demo Observatory is based on the principles of open collaboration, with the transparency of open source statistical software. It is a more methodological and broader version of the more developed and more specific Demo Music Observatory.
We are open for collaborations with representative international music organizations, representative national organizations, city-, regional- or provincial organizations, and business and scientific research organizations, and, because the music industry is largely made up from creative freelancer networks and microenterprises, even individual researchers (to a limited extent). We also hope to serve individual artists, technicians and managers very soon.
We are very grateful for Moon Moon Moon who explained to one of our hoped-for partner that making music industry websites is not easy. We got a lot of inspiration from their work and asked a soundtrack for our demonstration video about making an automated music industry website that also serves as the prototype-twin of this automated CCS observatory. We are certainly not there, but we hope to have a proof of concept.↩︎