The digital subsurface water-cooler

Back in August 2016 I told you about the Software Underground, an informal, grass-roots community of people who are into rocks and computers. At its heart is a public Slack group (Slack is a bit like Yammer or Skype but much more awesome). At the time, the Underground had 130 members. This morning, we hit ten times that number: there are now 1300 enthusiasts in the Underground!

If you’re one of them, you already know that it’s easily the best place there is to find and chat to people who are involved in researching and applying machine learning in the subsurface — in geoscience, reservoir engineering, and enything else to do with the hard parts of the earth. And it’s not just about AI… it’s about data management, visualization, Python, and web applications. Here are some things that have been shared in the last 7 days:

• News about the upcoming Software Underground hackathon in London.

• A new Udacity course on TensorFlow.

• Questions to ask when reviewing machine learning projects.

• A Dockerfile to make installing Seismic Unix a snap.

• Mark Zoback’s new geomechanics course.

It gets better. One of the most interesting conversations recently has been about starting a new online-only, open-access journal for the geeky side of geo. Look for the #journal channel.

Another emerging feature is the ‘real life’ meetup. Several social+science gatherings have happened recently in Aberdeen, Houston, and Calgary… and more are planned, check #meetups for details. If you’d like to organize a meetup where you live, Software Underground will support it financially.

We’ve also gained a website, softwareunderground.org, where you’ll find a link to sign-up in the Slack group, some recommended reading, and fantastic Software Underground T-shirts and mugs! There are also other ways to support the community with a subscription or sponsorship.

If you’ve been looking for the geeks, data-heads, coders and makers in geoscience and engineering, you’ve found them. It’s free to sign up — I hope we see you in there soon!

---

Slack has nice desktop, web and mobile clients. Check out all the channels — they are listed on the left:

The open subsurface stack

Two observations:

1. Agile has been writing about open source software for geology and geophysics for several years now (for example here in 2011 and here in 2016). Progress is slow. There are lots of useful tools, but lots of gaps too. Some new tools have appeared, others have died. Conclusion: a robust and trusted open stack is not going to magically appear.

2. People — some of them representing large corporations — are talking more than ever about industry collaboration. Open data platofrms are appearing all over the place. And several times at the DigEx conference in Oslo last week I heard people talk about open source and open APIs. Some organizations, notably Equinor, seem to really mean business. Conclusion: there seems to be a renewed appetite for open source subsurface software.

A quick reminder of what ‘open’ means; paraphrasing The Open Definition and The Open Source Definition in a sentence:

“Open data, content and code can be freely used, modified, and shared by anyone for any purpose.”

The word ‘open’ is being punted around quite a bit recently, but you have to read the small print in our business. Just as OpenWorks is not ‘open’ by the definition above, neither is OpenSpirit (remember that?), nor the Open Earth Community. (I’m not trying to pick on Halliburton but the company does seem drawn to the word, despite clearly not quite understanding it.)

The conditions are perfect

Earlier I said that a robust and trusted ‘stack’ (a collection of software that, ideally, does all the things we need) is not going to magically appear. What do I mean by ‘robust and trusted’? It goes far beyond ‘just code’ — writing code is the easy bit. It means thoroughly tested, carefully documented, supported, and maintained. All that stuff takes work, and work takes people and time. And people and time mean money.

Two more observations:

1. Agile has been teaching geocomputing like crazy — 377 people in the last year. In our class, the participants install a lot of Python libraries, including a few from the open subsurface stack: segyio, lasio, welly, and bruges. Conclusion: a proto-stack exists already, hundreds of users exist already, and some training and support exist already.

2. The Software Underground has over 1200 members (you should sign up, it’s free!). That’s a lot of people that care passionately about computers and rocks. The Python and machine learning communies are especially active. Conclusion: we have a community of talented scientists and developers that want to get good science done.

So what’s missing? What’s stopping us from taking open source subsurface tech to the next level?

Nothing!

Nothing is stopping us. And I’ve reached the conclusion that we need to provide care and feeding to this proto-stack, and this needs to start now. This is what the TRANSFORM 2019 unconference is going to be about. About 40 of us (you’re invited!) will spend five days working on some key questions:

• What libraries are in the Python ‘proto-stack’? What kind of licenses do they have? Who are the maintainers?

• Do we need a core library for the stack? Something to manage some basic data structures, units of measure, etc.

• What are we calling it, who cares about it, and how are we going to work together?

• Who has the capacity to provide attention, developer time, existing code, or funds to the stack?

• Where are the gaps in the stack, and which ones need to be filled first?

We won’t finish all this at the unconference. But we’ll get started. We’ll produce a lot of ideas, plans, roadmaps, GitHub issues, and new code. If that sounds like fun to you, and you can contribute something to this work — please come. We need you there! Get more info and sign up here.

---

Read the follow-up post >>> What’s happening at TRANSFORM?

---

Thumbnail photo of the Old Man of Hoy by Tom Bastin, CC-BY on Flickr.

Test driven development geoscience

Sometimes I wonder how much of what we do in applied geoscience is really science. Is it really about objective enquiry? Do we form hypotheses, then test them? The scientific method is largely a caricature — science is more accidental and more fun than a step-by-step recipe — but I think our field sometimes falls short of even basic rigour. Go and sit through a conference session on seismic attribute analysis some time and you'll see what I mean. Let's just say there's a lot of arm-waving and shape-ology. 

Learning from geeks

We've written before about the virtues of the software engineering community. Innovation has been so rapid recently, that I think it's a great place to find interpretation hacks like pair picking. Learning about and experiencing the amazing productivity of programmers is one of the reasons I think all scientists should learn to program (but not learn to be a programmer). You'll find out about concepts like version control, user-centered design, and test-driven development. Programmers embrace these ideas to a greater or lesser degree, depending on their goals and those of the project they're working on. But all programmers know them.

I'm especially into test-driven development at the moment. The idea is that before implementing a new module or feature, you write a test — a short program that gives the new thing some input, inspects the output, and compares it to a known answer. The first version of the code will likely fail the test. The idea is to refactor the code until it passes the test. Then you add that test to a suite that runs every time you build anything in the same project, so you know your new thing doesn't get broken by something else later. And you aren't tempted to implement features that weren't part of the test.

Fail — Refactor — Pass

Imagine test-driven development geology (or any other kind of geoscience). What would that look like?

• When planning wells, we often do write tests — they're called prognoses. But the comparison with the result is rarely formalized or quantified, especially outside the target zone. Once the well is drilled, it becomes data and we move on. No-one likes to dwell on the poorly understood or error-prone, but naturally that's where the greatest room for improvement is.  
• When designing a new seismic attribute, or embarking on a seismic processing project, we often have a vague idea of success in our heads, and that's about it. What if we explicitly defined an input test dataset, some wells or bits of wells, and set 'passing' performance criteria on those? "I won't interpret the reprocessed seismic until it improves those synthetic correlation coefficients by 40%."
• When designing a seismic survey, we could establish acceptable criteria for trace density, minimum offset, azimuth distribution, and recording time, then use these as a cost function to find the best possible survey for our dollars. Wait, perhaps we actually do this one. Is seismic acquisition unusually scientific? Or is it an inherently more linear problem?

What do you think? Can you see ways to define 'success' before you begin, then somewhat quantitatively compare your results with that? Ideas wanted!

Communicating geoscience

On Day 1 of the AAPG Annual Convention, I spent most of the morning in a special session entitled Communicating Our Science. I thought I'd share some of my thoughts on the subject. Please share yours in the comments!

This was primarily a panel discussion, but the convenors took several of questions from the audience of about 40 people. If the room had been smaller, and the audio system better-behaved, we might have had more of a conversation.

The panel consisted of three academics, two journalists, and a political lobbyist. While most of the panel had some direct experience in industry, there was a conspicuous and slightly mystifying absence of anyone currently working in industry. There was a less surprising but possibly more troubling absence of anyone representing 'the public' (whoever they are). The panelists were:

• Jim Reilly, Dean of Science and Technology Development, American Public University System
• Michael Zehr, Federal Policy Advisory, Consumer Energy Alliance
• Heather Saucier, Correspondent, AAPG Explorer magazine
• Jane Whaley, Editor in Chief, GeoExPro Magazine
• Donald Paul, Professor of Engineering, Chair of Energy Resources, USC
• Iain Stewart, Professor of Geosciences Communication, University of Plymouth

Questions covered topics like "Where can I get information about energy issues?" (the EIA is a good start) to "How should I answer emotional questions about fracking?"(Honestly... and send people to Frack-Land). Most of the answers focused on engaging with the public, or with the press. But I'm most interested in how our own community communicates with itself, so the question I wanted to ask (I wasn't chosen by the chairs) was this:

The panel advises industry to engage with the public and communicate with authenticity and transparency, and I agree completely. But my perception is that we don't model this type of communication when we talk to each other, never mind the public — companies are wary of sharing methods and data, and we're most comfortable with controlled, one-way communication like talks, papers, and panel discussions. Does the panel agree, and if so, does it have any recommendations for improving how our technical community talks to itself? 

I do love the concept of sessions like this — we absolutely need more of them. But I think we need to think more about the purpose, and what we can get out of them. For example, no-one was capturing the proceedings, as far as I could tell (and a video would certainly be the wrong way to do that here). And there was no way to circle back to the crowd for a reflection on the panel's responses. We can do better.

That said, there were plenty of nuggets of wisdom from the panel. One of my favourites was a remark from Iain Stewart: "I don't think we should be focusing on increasing the public's scientific literacy, we should be focusing on building trust." At the very end, the panel were asked for advice for AAPG and the professional community:

• AAPG should build a repository of links, blogs, and FAQs about geoscience — Jim Reilly
• We need more data to support our reporting and outreach — Michael Zehr
• Technical societies should offer writing courses and seminars — Heather Saucier (here's one :)
• Industry has to be more proactive with public engagement — Jane Whaley
• Public engagement is not an option anymore, it's a core skill — Donald Paul
• Communicators should embrace the human side of science with enthusiasm — Iain Stewart

I could get behind all of those. What about you?

A long weekend of creative geoscience computing

The Rock Hack is in three weeks. If you're in Houston, for AAPG or otherwise, this is going to be a great opportunity to learn some new computer skills, build some tools, or just get some serious coding done. The Agile guys — me, Evan, and Ben — will be hanging out at START Houston, laptops open, all say 5 and 6 April, about 8:30 till 5. The breakfast burritos and beers are on us.

Unlike the geophysics hackathon last September, this won't be a contest. We're going to try a more relaxed, unstructured event. So don't be shy! If you've always wanted to try building something but don't know where to start, or just want to chat about The Next Big Thing in geoscience or technology — please drop in for an hour, or a day.

Here are some ideas we're kicking around for projects to work on:

• Sequence stratigraphy calibration app to tie events to absolute geologic time and to help interpret systems tracts.
• Wireline log 'attributes'.
• Automatic well-to-well correlation.
• Facies recognition from core.
• Automatic photomicrograph interpretation: grain size, porosity, sorting, and so on.
• A mobile app for finding and capturing data about outcrops.
• An open source basin modeling tool.

Short course

If you feel like a short course would get you started faster, then come along on Friday 4 April. Evan will be hosting a 1-day course, leading you through getting set up for learning Python, learning some syntax, and getting started on the path to scientific computing. You won't have super-powers by the end of the day, but you'll know how to get them.

The course includes food and drink, and lots of code to go off and play with. If you've always wanted to get started programming, this is your chance!

May linkfest

The monthly News post wasn't one of our most popular features, so it's on the shelf. Instead, I thought I'd share all the most interesting, quirky, mind-blowing, or just plain cool things I've spotted on the web over the last month.

– Do not miss this. One of them stands out above all the others. If you like modern analogs and satellite imagery, you're going to love Google Earth Engine. I've started a list of geologically interesting places to visit — please add to it!

– More amazing images. I'll never get bored of looking at gigapans, and Callan Bentley's are among the best geological ones. I especially like his annotated ones.

– Classic blog. Greg Gbur writes one of the best physics blogs, and his focus is on optics, so there's often good stuff there for geophysicists. This post on Chladni patterns is pure acoustic goodness and well worth a slow read. 

– New geoscience blog. Darren Wilkinson is a young geoscientist in the UK, and writes a nice geeky blog about his research. 

– Brilliant and simple. Rowan Cockett is a student at UBC, but builds brilliant geological web apps on the side. He has a knack for simplicity and his latest creation makes stereonets seem, well, simple. Impressive. 

– New magazine. Kind of. There's not enough satire or ragging in the petroleum press, so it's refreshing to hear of Proved Plus Probable, a fairly wacky weekly online rag emanating from Calgary [thanks to Dan for the tip!]. Top headline: Legendary geologist invents new crayons. 

– Counter-factual geology. I love these pictures of an imagined ring around earth.

– Never buy graph paper again. Make some just how you like it!

– Bacon. It was a revelation to find that some rocks look just like bacon.

That's it! I share most of this sort of thing on Twitter. Really useful stuff I tend to stick on my pinboard — you're welcome to browse. If you have a geological or geeky bookmark collection, feel free to share it in the comments!

Here comes GeoConvention 2013

Next week Matt and I are heading to the petroleum capital of Canada for the 2013 GeoConvention. There will be 308 talks, 125 posters, over 4000 attendees, 100 exhibiting companies, and at least 2 guys blogging their highlights off.

My picks for Monday

Studying the technical abstracts ahead of time is the only way to make the most of your schedule. There are 9 sessions going on at any given time, a deep sense of FOMO has already set in. These are the talks I have decided on for Monday: 

Seismics for unconventionals

I watched Carl Reine from Nexen give a talk two years ago where he deduced a power-law relationship characterizing natural fracture networks in the Horn River shale. He will show how integrating such fracture intensity patterns with inversion models yields a powerful predictor of frackability, and uses microseismic to confirm it.

On a related note, and also from the Horn River Basin, Andreas Wuestefeld will show how microseismic can be used to identify fluid drainage patterns from microseismic data. Production simulation from an actual microseismic experiment. Numerical modeling, and physical experiment inextricably linked. I already love it.

Forward models and experimental tests

One is a design case study for optimizing interpolation, the other is a 3D seismic geometry experiment, the third is a benchtop physical fracture model made out of Plexiglass and resin.

Broadband seismic

Gets to the point of what hinders seismic resolution, and it does something about it through thoughtful design. This is just really nice looking data, two talks, same author: a step change, and impact of broadband. 

Best title award 

Goes to Forensic chemostratigraphy. Gimmicky name or revolutionary concept? You can't always judge a talk by the title, or the quality of the abstract. But it's hard not to. What talks are on your must-see list this year?

A really good conversation

Matt and I are hosting an unsession on the morning of Tuesday 7 May. It will be structured, interactive, and personal. The result: a ranked list of the most pressing problems facing the upstream geoscientists, especially in those hard to reach places between the disciplines. This is not a session where you sit and listen. Everyone will participate. We will explore questions that matter, connect diverse perspectives, and, above all, capture our collective knowledge. It might be scary, it might be uncomfortable, it might not be for you. But if you think it is, bring your experience and individuality, and we will do that thing called integration. We can only host 60 people, so if you don't want to be turned away, arrive early to claim a spot. We start at 8 a.m. in Telus 101/102 on the main floor of the north building.

What is an unsession?

Yesterday I invited you (yes, you) to our Unsolved Problems Unsession on 7 May in Calgary. What exactly will be involved? We think we can accomplish two things:

1. Brainstorm the top 10, or 20, or 50 most pressing problems in exploration geoscience today. Not limited to but focusing on those problems that affect how well we interface — with each other, with engineers, with financial people, with the public even. Integration problems.
2. Select one or two of those problems and solve them! Well, not solve them, but explore ways to approach solving them. What might a solution be worth? How many disciplines does it touch? How long might it take? Where could we start? Who can help?

There are bright, energetic young people out there looking for relevant problems to work on towards a Master's or PhD. There are entrepreneurs looking for high-value problems to create a new business from. And software companies looking for ways to be more useful and relevant to their users. And there is more than one interpreter wishing that innovation would speed up a bit in our industry and make their work a little — or a lot — easier. 

We don't know where it will lead, but we think this unsession is one way to get some conversations going. This is not a session to dip in and out of — we need 4 hours of your time. Bring your experience, your uniqueness, and your curiosity.

Let's reboot our imaginations about what we can do in our science.

An invitation to a brainstorm

Who of us would not be glad to lift the veil behind which the future lies hidden; to cast a glance at the next advances of our science and at the secrets of its development during future centuries? What particular goals will there be toward which the leading [geoscientific] spirits of coming generations will strive? What new methods and new facts in the wide and rich field of [geoscientific] thought will the new centuries disclose?

— Adapted from David Hilbert (1902). Mathematical Problems, Bulletin of the American Mathematical Society 8 (10), p 437–479. Originally appeared in in Göttinger Nachrichten, 1900, pp. 253–297.

Back at the end of October, just before the SEG Annual Meeting, I did some whining about conferences: so many amazing, creative, energetic geoscientists, doing too much listening and not enough doing. The next day, I proposed some ways to make conferences for productive — for us as scientists, and for our science itself. 

Evan and I are chairing a new kind of session at the Calgary GeoConvention this year. What does ‘new kind of session’ mean? Here’s the lowdown:

The Unsolved Problems Unsession at the 2013 GeoConvention will transform conference attendees, normally little more than spectators, into active participants and collaborators. We are gathering 60 of the brightest, sparkiest minds in exploration geoscience to debate the open questions in our field, and create new approaches to solving them. The nearly 4-hour session will look, feel, and function unlike any other session at the conference. The outcome will be a list of real problems that affect our daily work as subsurface professionals — especially those in the hard-to-reach spots between our disciplines. Come and help shed some light, room 101, Tuesday 7 May, 8:00 till 11:45.

What you can do

• Where does your workflow stumble? Think up the most pressing unsolved problems in your workflows — especially ones that slow down collaboration between the disciplines. They might be organizational, they might be technological, they might be scientific.
• Put 7 May in your calendar and come to our session! Better yet, bring a friend. We can accommodate about 60 people. Be one of the first to experience a new kind of session!
• If you would like to help host the event, we're looking for 5 enthusiastic volunteers to play a slightly enlarged role, helping guide the brainstorming and capture the goodness. You know who you are. Email hello@agilegeoscience.com. 

5 ways to kickstart an interpretation project

Last Friday, teams around the world started receiving external hard drives containing this year's datasets for the AAPG's Imperial Barrel Award (IBA for short). I competed in the IBA in 2008 when I was a graduate student at the University of Alberta. We were coached by the awesome Dr Murray Gingras (@MurrayGingras), we won the Canadian division, and we placed 4th in the global finals. I was the only geophysical specialist on the team alongside four geology graduate students.

Five things to do

Whether you are a staff geoscientist, a contractor, or competitor, it can help to do these things first:

1. Make a data availability map (preferably in QGIS or ArcGIS). A graphic and geospatial representation of what you have been given.
2. Make well scorecards: as a means to demonstrate not only that you have wells, but what information you have within the wells.
3. Make tables, diagrams, maps of data quality and confidence. Indicate if you have doubts about data origins, data quality, interpretability, etc.
4. Background search: The key word is search, not research. Use Mendeley to organize, tag, and search through the array of literature
5. Use Time-Scale Creator to make your own stratigraphic column. You can manipulate the vector graphic, and make it your own. Much better than copying an old published figure. But use it for reference.

All of these things can be done before assigning roles, before saying who needs to do what. All of this needs to be done before the geoscience and the prospecting can happen. To skirt around it is missing the real work, and being complacent. Instead of being a hammer looking for a nail, lay out your materials, get a sense of what you can build. This will enable educated conversations about how you can spend your geoscientific manpower, division of labour, resources, time, etc.

Read more, then go apply it 

In addition to these tips for launching out of the blocks, I have also selected and categorized blog posts that I think might be most relevant and useful. We hope they are helpful to all geoscientists, but especially for students. Visit the Agile blog highlights list on SubSurfWiki.

I wish a happy and exciting IBA competition to all participants, and their supporting university departments. If you are competing, say hi in the comments and tell us where you hail from.