25 Jul The Rise of the Platform Solution
Platform solutions save time and money and improve business decision quality. And now there are even specific platforms for oil and gas.
I’ve reached into the archives for this article, which was originally published on August 16, 2021.
The Origins of Platforms
By platform, I don’t mean off-shore—I mean a computerized business system.
I didn’t appreciate it at the time, but my first exposure to platform thinking was in 1984 on my first corporate job at a big oil company where I supported a computer system called CORPS. It was written in PL/1 (a programming language) and ran on an IBM S360 TSO mainframe system at 1 am (yes, I carried a pager and frequently was jolted awake at 2 am when the system didn’t run correctly, usually my fault). Big companies love abbreviations—I can’t recall what exactly CORPS stood for (Corporate Reporting System, possibly) but I remember precisely why it existed: to save data center time and cost in mounting, spinning and dismounting magnetic tapes.
CORPS was a kind of middleman system. At the time, in an era before SAP, this big oil company had a handful of major commercial business systems that handled different aspects of petroleum product movement (purchases, sales, volumes, lifts, loadings, inventories for wholesale, retail, commercial and industrial markets) which individually fed data to many other systems (margin analysis, financials, customer accounts). The dozens of individual data feeds from one system to another (load magnetic tape on tape drive, copy data to another tape, dismount tapes) created a literal Gordian Knot of integrations. A failure in any one brought the whole system to its knees.
CORPS was an attempt to solve for this problem—all the inputs fed into one gigantic master file which then transformed the data, and generated all the individual data feeds. It was dramatically faster than running all the data transfers individually.
With hindsight, CORPS solved a many-to-many problem. Many data inputs going to many data outputs creates huge cost as each data supplier needs to maintain an individual connection with each data consumer. A change to any one system can have a ripple effect on many other systems.
Modern digital platforms are very good at solving the many-to-many problem. For example, trading platforms facilitate buyers and sellers to find one another and transact. Amazon matches many suppliers of goods with many customers. AirBnB matches parties holding available accommodations with travellers needing a short term place to stay. Uber matches cars and drivers with customers. Platforms can often capture network effects by connecting very large numbers of counterparties.
Solve for Many to Many
As with CORPS, economic impacts are magnified when the problem of many distinct data sources feeding many data consumers is solved. Upstream oil and gas features a rich diversity of commercial software packages solving for very specific analytic problems, which results in an abundance of data sources. In addition, engineers rely on ERP systems, land systems, mapping software, paper binders, PDFs, data lakes, spreadsheets and historians to supply data. Other data sources include sensors, robots and edge computing devices, web services and RSS feeds, and SCADA systems.
Resources businesses also have numerous unique internal uses for the data (such as well planning, capital budgeting, geologic analysis, environmental studies, environmental reporting, compliance and financials) which can take the form of spreadsheets, commercial software products, analytics and visualisation software, and increasingly, machine learning algorithms and machines directly.
But what truly distinguishes platform solutions from other solutions are the presence of features on the platform that allow platform users to define their own ways of doing business. Here are a handful of the kinds of features that should be on the shortlist of criteria for choosing a platform for your upstream business.
Enable Rapid and Flexible Expansion
The software code that enables access to the data in a data source (such as a spreadsheet) is generalised so that it can access data from any similar data source (another spreadsheet) and has a user interface that does not require the user to be the original programmer. These application programming interfaces (or APIs) are the magic that make platforms super powerful—once written APIs are reusable, they provide a kind of insulation between the data source (which can change on its own schedule) and the consumer of the data (which can also change). APIs are scalable in their own right.
I look for platforms that feature extensive and constantly growing libraries of these APIs that enable both data sources and data outputs.
Render Data Truly Useful
Holding all of the data from the various sources so that they can be consumed by the users creates a data management problem. CORPS was rather unsophisticated in that regard in that the system simply created a nightly file of master data. Modern platforms deal with this challenge by incorporating their own data base solutions with meta data management, naming conventions, data structures, and the myriad elements required to hold and eventually manipulate petabytes of data.
One of the important features of CORPS was unit harmonization. Some feeder sources predated Canadian adoption of the metric system for weights and volumes, and product volumes were in tons and cubic feet, but reporting systems wanted only metric tonnes and cubic meters. Some measures were corrected at source for thermal expansion (the volume of petroleum expands as it heats up) and others needed correction to a standard. Some dates were in format YYMMDD, and needed to be transformed to YYYYMMDD.
It turns out that data harmonisation is still quite commonplace. Feeder system A refers to a well using its naming standard, but that standard is different from Feeder system B. The platform needs to harmonize the data, provide an audit trail about the changes to the data, and permit traceability forward and backward from feeder to consumer.
These services are called taxonomy, they enable this kind of data sophistication, and like APIs, they are reusable and scalable. Good platforms invest extensively in taxonomy capabilities.
Allow for Rapid Low Cost Growth
One of CORPs problems was its lack of scalability. It only ran on one specifically set up mainframe. Physical devices like tape drives were hard named in the job control language (or JCL). It had no test bed (mainframes are expensive) so changes might not be fully be debugged before they were introduced into production (as I discovered frequently with the 2 am pager). Adding a new interface took 8 weeks of work.
Platforms get around these limitations by designing for scalability. They run on industry standard cloud platforms that offer near infinite growth capacity for both data storage and compute cycles. One of the cloud instances is probably the test bed so that changes can be thoroughly tested out. Instead of Joe Programmer having to imagine a testing plan for every change, they incorporate sophisticated testing tools that run through thousands of different kinds of variations. They use responsive browser technology so that any device anywhere anytime can access the platform.
Protect The Asset
CORPS had no security. I don’t think cyber was even a thing back then. Yes, there were passwords to get onto systems, but I have no recollection at all of concepts like authentication of a data source, and authorization of a device to supply data. Data was not encrypted because all work took place inside the company computer system, there was no internet access (yet) and encryption imposed a compute cycle cost.
Today, encryption of data from end to end, and other security concepts, are top of mind for managers and leaders. Platform solutions are particularly at risk because they represent a single point of failure. Modern platform systems invest extensively in hardening their systems to repel the inevitable cyber attack.
Enable Citizen Programmers
CORPS offered little value added beyond some data standardisation. Everything was hand-coded by a scarce programmer using an arcane software language (PL/1 in my case). Modern platforms include their own kind of easy to learn programming language so that ordinary users can code up unique and specific solutions to highly nuanced problems.
- Does your process call for Finance to certify a financial projection (for cash flow reasons) before your engineers can embark on a well work over? No problem, just code up the workflow to your method.
- Does your monthly reporting package need to transform some late-arriving data on midstream throughput from your gas processor? No problem, just build the routine you need and trigger it to execute on receipt of the data.
Programmability opens up the possibility of third party applications being written for the platform. These applications then unlock new business models (as an app provider, for example) new technology integrations (such as incorporating augmented and virtual reality) new commercial models (using advertising, crypto currency payment methods, or subscriptions) data streaming (for transactions) and new off-platform integrations (such as merging specialised inaccessible datasets). I think of this as akin to the App Store from Apple, which has created billions of dollars of incremental value above the iOS platform.
Advanced platforms build in support for third parties to help evolve the value of the platform, including concepts like app stores, supplier certifications, programming standards, security protocols and data privacy standards.
Deepen Your Analytics
With so much data at hand, platforms can offer layers of algorithms, models, data science, learning services, visualizations and manipulations that are too expensive for individual feeder systems, or not valuable without the significant data volumes that platforms provide.
Show Me The Money
As we can see from Amazon, Apple, Uber, and AirBnB, platform solutions are the future. They allow companies to implement change much faster. Hidden overhead costs from excessive data handling, hoarding, and manipulation get progressively squeezed out. Decision quality improves because there is more time for insight and analysis. Innovation can blossom because of the high quality of underlying data. New technologies such as machine learning and augmented reality finally have a fighting chance to succeed. New business models emerge.
A Working Example
For a great working example of a platform in action, check out Datagration and their PetroVisor platform. It fulfils all of the kinds of features that I expect to see in world class platform technology.
Platforms Are The Future
The roots of platform systems date back many years, but today’s versions are simply too valuable to ignore, and with the pressures to solve for problems like carbon tracing, industry specific platforms will grow in prominence and affordability.
Check out my latest book, ‘Carbon, Capital, and the Cloud: A Playbook for Digital Oil and Gas’, available on Amazon and other on-line bookshops.
You might also like my first book, Bits, Bytes, and Barrels: The Digital Transformation of Oil and Gas’, also available on Amazon.
Take Digital Oil and Gas, the one-day on-line digital oil and gas awareness course on Udemy.