2018 retrospective

It’s almost the end of another trip around the sun. I hope it’s been kind to you. I mean, I know it’s sometimes hard to see the kindness for all the nonsense and nefariousness in <ahem> certain parts of the world, but I hope 2018 at least didn’t poke its finger in your eye, or set fire to any of your belongings. If it did — may 2019 bring you some eye drops and a fire extinguisher.

Anyway, at this time of year, I like to take a quick look over my shoulder at the past 12 months. Since I’m the over-sharing type, I like to write down what I see and put it on the Internet. I apologize, and/or you’re welcome.

Top of the posts

We’ve been busier than ever this year, and the blog has taken a bit of a hit. In spite of the reduced activity (only 45 posts, compared to 53 last year), traffic continues to grow and currently averages 9000 unique visitors per month. These were the most visited posts in 2018:

• Big open data, or is it? — about the amazing Volve dataset released by Equinor earlier this year. Don’t miss the follow-up post about its licensing: Volve: not open after all, in which I discuss the license that Equinor chose.

• x lines of Python: contour maps — making nice contour maps in Python (strictly a 2017 post, but almost all of its traffic came in 2018).

• What is scientific computing? — the emergence of the ‘digital geoscientist’.

• Results from the AAPG machine learning unsession — what happened in Salt Lake City.

• Jounce, crackle and pop — derivatives of displacement you never knew you cared about.

Last December’s post, No more rainbows, got more traffic this year than any of these posts. And, yet again, k is for wavenumber got more than any. What is it with that post??

Where in the world?

Every year I take a look at where our people are reading the blog from (according to Google). We’ve travelled more than usual this year too, so I’ve added our various destinations to the map… it makes me realize we’re still missing most of you.

1. Houston (number 1 last year)

2. London (up from 3)

3. Calgary (down from 2)

4. Stavanger (6)

5. Paris (9)

6. New York (—)

7. Perth (4)

8. Bangalore (—)

9. Jakarta (—)

10. Kuala Lumpur (8)

Together these cities capture at least 15% of our readship. New York might be an anomaly related to the location of cloud infrastructure there. (Boardman, Oregon, shows up for the same reason.) But who knows what any of these numbers mean…

Work

People often ask us how we earn a living, and sometime I wonder myself. But not this year: there was a clear role for us to play in 2018 — training the next wave of digital scientists and engineers in subsurface.

• We continued the machine learning project on GPR interpretation that we started last year.

• We revived Pick This and have it running on a private corporate cloud at a major oil company, as well as on the Internet.

• We have spent 63 days in the classroom this year, and taught 325 geoscientists the fundamentals of Python and machine learning.

• Apart from the 6 events of our own that we organized, we were involved in 3 other public hackathons and 2 in-house hackathons.

• We hired awesome digital geologist Robert Leckenby (right) full time. 

The large number of people we’re training at the moment is especially exciting, because of what it means for the community. We spent 18 days in the classroom and trained 139 scientists in the previous four years combined — so it’s clear that digital geoscience is important to people today. I cannot wait to see what these new coders do in 2019 and beyond!

The hackathon trend is similar: we hosted 310 scientists and engineers this year, compared to 183 in the four years from 2013 to 2017. Numbers are only numbers of course, but the reality is that we’re seeing more mature projects, and more capable coders, at every event. I know it’s corny to say so, but I feel so lucky to be a scientist today, there is just so much to do.

Cheers to you

Agile is, as they say, only wee. And we all live in far-flung places. But the Intertubes are a marvellous thing, and every week we meet new people and have new conversations via this blog, and on Twitter, and the Software Underground. We love our community, and are grateful to be part of it. So thank you for seeking us out, cheering us on, hiring us, and just generally being a good sport about things.

From all of us at Agile, have a fantastic festive season — and may the new year bring you peace and happiness.

2017 retrospective

Another year pulls on its winter boots and prepares to hurry through the frigid night to wherever old years go to die. From a purely Agile point of view, putting aside all the odious nonsense going on in the world for a moment, it was a good year here at Agile, and I hope it was for you too. If not — if you were unduly affected by any of the manifold calamities in 2017 — then we wish you the best and hope life bounces back with renewed vigour in 2018.

>>>
A reproducible festive card for you, made from a well-
log and a bunch of random numbers. Make your own. 

---

It's that time when I like to self-indulgently glance back over the last twelve months — both on the blog and elsewhere in the Agile universe. Let's start with the blog...

The most popular posts

We should top 52 posts this year (there's just something about the number 52). Some of them do little more than transmit news, events and such, but we try to bring you entertainment and education too. Just no sport or weather. These were our most visited posts in this year:

• Machine learning meets seismic interpretation — Evan's highlights from an analytics session at EAGE in Paris.
• Machine learning and analytics in geoscience — what happened at the analytics workshop at EAGE in Paris.
• Attribution is not permission — all about that time when Elsevier published a horrible book about reservoir analysis.
• No more rainbows! — last week's post about rainbow colourmaps broke all our traffic records.
• The new reality — how the future is not going to be like the past. Not in petroleum geoscience anyway.

As usual though, the most popular page on the site is k is for wavenumber, the 2012 post that keeps on giving. The other perennials are Well tie workflow, What is anisotropy? and What is SEG Y? 

Engagement

We love getting comments! Most people tend to chime in via Twitter or LinkedIn, but we get quite a few on the blog. Indeed, the posts listed above got more than 60 comments between them. The following were the next most commented upon:

• Organizing spreadsheets — how to make a spreadsheet more useful by making it more like a database.
• Isn't everything on the internet free? — a follow-up post to Attribution is not permission (see above).
• More precise SEG-Y and SEG-Y Rev 2 again — a pair of posts looking at the new SEG-Y standard.

Where is everybody?

1. Houston (about 6.6% of you)
2. Calgary (4.8%)
3. London (3.3%)
4. Perth (1.8%)
5. Moscow (1.3%)
6. Stavanger (1.2%)
7. Rio de Janiero (1.1%)
8. Kuala Lumpur (1.0%)
9. Paris (1.0%)
10. Aberdeen (0.9%)

Work

We're fortunate to have had a good year at Agile. I won't beat our drum too hard, but here's a bit of what we've been up to:

• We're doing a machine learning project on GPR interpretation.
• We finished a machine learning lithology prediction project for Canstrat.
• Matt did more seep and DHI mapping on Canada's Atlantic margin.
• It was a good year for hackathons, with over 100 people taking part in 2017.
• Agile Libre brought out a new book, 52 More Things... Palaeontology.
• We hired awesome data scientist Diego Castañeda (right) full time. 

Thank you

Last but far from least — thank you. We appreciate your attention, one of the most precious resources you have. We love writing useful-and/or-interesting stuff, and are lucky to have friends and colleagues who read it and push us to do more, and a bit better than before. It would be a chore if it wasn't for your readership.

All the best for this Yuletide season, and for a peaceful New Year. Cheers!

What now?

Times are rock hard in industry right now.

If you have a job, you're lucky — you have probably already survived one round of layoffs. There will likely be more, especially when the takeovers start, which they will. I hope you survive those too. 

If you don't have a job, you probably feel horrible, but of course that won't get you anywhere. I heard one person call it an 'involuntary sabbatical', and I love that: it's the best chance you'll get to re-invent, re-learn, and find new direction. 

If you're a student, you must be looking out over the wasteland and wondering what's in store for you. What on earth?

More than one person has asked me recently about Agile. "You got out," they say, "how did you do it?" So instead of bashing out another email, I thought I'd blog about it.

Consulting in 2015

I didn't really get out, of course, I just quit and moved to rural Nova Scotia.

Living out here does make it harder to make a living, and things on this side of the fence, so to speak, are pretty gross too I'm afraid. Talking to others at SEG suggested that I'm not alone among small companies in this view. A few of the larger outfits seem to be doing well: IKON and GeoTeric for instance, but they also have product, which at least offers some income diversity. 

Agile started as a 100% bootstrapped effort to be a consulting firm that's more directly useful to individual professional geoscientists than anyone else. Most firms target corporate accounts and require permission, a complicated contract, an AFE, and 3 months of bureaucracy to hire. It turns out that professionals are unable or unwilling to engage on that lower, grass-roots level, though — turns out almost everyone thinks you actually need permission, contracts, AFEs, etc, to get hired in any capacity, even just "Help me tie this well." So usually we are hired into larger, longer-term projects, just like anyone else.

I still think there's something in this original idea — the Uberification of consulting services, if you will — maybe we'll try again in a few years.

But if you are out of work and were thinking of getting out there as a consultant... I'm an optimistic person, but unless you are very well known (for being awesome), it's hard for me to honestly recommend even trying. It's just not the reality right now. We've been lucky so far, because we work in geothermal and government as well as in petroleum, but oil & gas was over half our revenue last year. It will be about 0% of it this year, maybe slightly less.

The transformation of Agile

All of which is to explain why we are now, since January, consciously and deliberately turning ourselves into a software technology R&D company. The idea is to be less dependent on our dysfunctional industry, and less dependent on geotechnical work. We build new tools for hard problems — data problems, interpretation problems, knowledge sharing problems. And we're really good at it.

We hired another brilliant programmer in August, and we're all learning more every day about our playground of scientific computing and the web — machine learning, cloud services, JavaScript frameworks, etc. The first thing we built was modelr.io, which is still in active development. Our latest project is around our tool pickthis.io. I hope it works out because it's the most fun I've had on a project in ages. Maybe these projects spin out of Agile, maybe we keep them in-house.

So that's our survival plan: invent, diversify, and re-tool like crazy. And keep blogging.

F**k it

Some people are saying, "things will recover, sit it out" but I think that's awful — the very worst — advice. I honestly think your best bet right now* is to find an accomplice, set aside 6 months and some of your savings, push everything off your desk, and do something totally audacious. 

Something you can't believe no-one has thought of doing yet.

Whatever it was you just thought of — that's the thing.

You might as well get started.

---

* Unless you have just retired, are very well connected in industry, have some free time, and want to start a new, non-commercial project that will profoundly benefit the subsurface community for the next several decades at least. Because I'd like to talk to you about another audacious plan...

Your best work(space)

Doing your best work requires placing yourself in the right environment. For me, I need to be in an uncluttered space, free from major distractions, yet close enough to interactions to avoid prolonged isolation. I also believe in surrounding yourself with the energetic and inspired people, if you can afford such a luxury.

The model workspace

My wife an I are re-doing our office at home. Currently mulling over design ideas, but websites and catalogs only take me so far. I find they fall short of giving me the actual look and feel of a future space. To cope, I have built a model using SketchUp, catering to my geeky need for spatial visualization. It took me 35 minutes to build the framework using SketchUp: the walls, doors and closets and windows. Now, it's taking us much longer to design and build the workspace inside it. I was under the impression that, just as in geoscience, we need models for making detailed descisions. But perhaps, this model is complicating or delaying us getting started. Or maybe we are just being picky. Refined tastes.

This is a completely to-scale drafting of my new office. It is missing some furniture, but the main workspace is shown on the left wall; a large, expansive desk to house (up to) two monitors, two chairs, and two laptops. The wide window sill will be fitted with bench cushions for reading. Since we want a built-in look, it makes sense construct a digital model to see how the components line up with other features in the space. 

More than one place to work 

So much of what we do in geoscience is centered around effectively displaying information, so it helps to feel fresh and inspired by the environment beyond the desktop. Where we work affects how we work. Matt and I have that luxury of defining our professional spaces, and we are flexible and portable enough to work in a number of settings. I like this.

There is a second place to go to when I want to get out of the confines of my condo. I spend about 30 hours a month at a co-working space downtown. The change in scenery is invigorating. I can breathe the same air as like-minded entrepreneurs, freelancers, and sprouters of companies. I can plug into large monitors, duck into a private room for a conference call, hold a meeting, or collaborate with others. Part of what makes an office is the technology, the furniture, the lighting, which is important. The other part of a workspace is your relationship and interaction to other people and places; a sense of community.

What does your best work space look like? Are you working there now?

What technology?

This is my first contribution to the Accretionary Wedge geology themed community blog. Charles Carrigan over at Earth-like Planet is hosting this months topic where he posts the question, "how do you perceive technology impacting the work that you do?" My perception of technology has matured, and will likely continue to change, but here are a few ways in which technology works for us at Agile. 

My superpower

I was at a session in December where one of the activities was to come up with one (and only one) defining superpower. A comic-bookification of my identity. What is the thing that defines you? The thing that you are or will be known for? It was an awkward experience for most, a bold introspection to quickly pull out a memorable, but not too cheesy, superpower that fit our life. I contemplated my superhuman intelligence, and freakish strength... too immodest. The right choice was invisibility. That's my superpower. Transparency, WYSIWYG, nakedness, openness. And I realize now that my superpower is, not coincidentally, aligned with Agile's approach to technology. 

For some, technology is the conspicuous interface between us and our work. But conspicuous technology constrains your work, ordains it even. The real challenge is to use technology in a way that makes it invisible. Matt reminds me that how I did it isn't as important as what I did. Making the technology seem invisible means the user must be invisible as well. Ultimately, tools don't matter—they should slip away into the whitespace. Successful technology implementation is camouflaged. 

I is for iterate

Technology is not a source of ideas or insights, such as you'd find in the mind of an experienced explorationist or in a detailed cross-section or map. I'm sure you could draw a better map by hand. Technology is only a vehicle that can deliver the mind's inner constructs; it's not a replacement for vision or wisdom. Language or vocabulary has nothing to do with it. Technology is the enabler of iteration. 

So why don't we iterate more in our scientific work? Because it takes too long? Maybe that's true for a hand-drawn contour map, but technology is reducing the burden of iteration. Because we have never been taught humility? Maybe that stems from the way we learned to learn: homework assignments have exact solutions (and are done only once), and re-writing an exam is unheard of (unless you flunked it the first time around).

What about writing an exam twice to demonstrate mastery? What about reading a book twice, in two different ways? Once passively in your head, and once actively—at a slower pace, taking notes. I believe the more ways you can interact with your media, data, or content, the better work will be done. Students assume that the cost required to iterate outweighs the benefits, but that is no longer the case with digital workflows. Embracing technology's capacity to iterate seemlessly and reliably is what a makes a grand impact in our work.

What do we use?

Agile strives to be open as a matter of principle, so when it comes to software we go for open source by default. Matt wrote recently about the applications and workstations that we use. 

Two hundred posts

The petrophysics cheasheet was one of our most popular posts

My post on Tuesday was the two hundredth post on our blog, which we started 19 months ago in November 2010. Though we began with about 15 posts per month, we have settled down to a rate of 7 or 8 posts per month, which feels sustainable. At this rate, it will be at least a year before we hit 300.

We hit 100 posts on 21 June last year, after only 222 days. In the 358 days since then we've had about 41 700 visits from 24 500 people in 152 countries. The most popular content is a little hard to gauge because of the way we run every post over the home page for a couple of weeks, but from the most recent 100 posts, the favourites are (in descending pageview order):

• G is for Gather (6 comments)
• Petrophysics cheatsheet (5)
• Why petrophysics is hard (5)
• I is for Integrated trace (7)
• Are you a poet or a mathematician? (13)
• Pseudogeophysics (8)
• Wherefore art thou, Expert? (12)
• Geophysical stamps 3: Geophone (10)
• The Agile toolbox (11)
• Stop waiting for permission to knock someone's socks off (10)

Someone asked recently how long our posts take to write. It varies quite a bit, especially if there are drawings or other graphics, but I think the average is about 4 hours, perhaps a little more. Posts follow an idea–draft–hack–review–publish process, and this might be months long: we currently have 52 draft posts in the pipeline! Some may never make it out...

We'd love to have some other voices on the site, so if you feel strongly about something in this field, or would like the right to reply to one of our opinion pieces, please get in touch. Or start a blog!

One hundred posts

Yesterday Evan posted the 100th article on the blog. Not all of them have been technical, though most are. A few were special cases, hosting the popular Where on (Google) Earth game for example. But most have been filled with geoscience goodness... at least we think so. We hope you do too.

One hundred posts isn't exactly earth-shattering, but we're proud of our work and thought we'd share some some of our greatest hits. We have our favourites, naturally. I really liked writing What is unconventional, and thought it was quite original. And I love yesterday's post, which is Evan's favourite too.

We could look at the most commented (not counting WOGEs, which always get lots of comments). The most comments were garnered by Why we should embrace openness, which got eight, and only two of those were from Evan and I. Every comment gives us warm, fuzzy feelings and it's really why we do this: a big Thank You to all our commenters, especially the serial scribes j, Richie B, Reid, Brian, and Tooney—you are awesome. Basic cheatsheet got nine comments, but four of them were from us: we do try to respond to every comment. 

It's a little harder to tell which article is the most read. There's a bias through time, since older pages have been up longer. And the front page gets most of the traffic, and each article gets a spell as the top story, but we don't track which articles are up when that page is visited. 

The most visited page is Evan's brilliant Rock physics cheatsheet; the PDF is also the most downloaded file. This is good because Evan poured his heart into building that thing. The next most popular page is The scales of geoscience, which benefitted hugely from being tagged in the social bookmarking site reddit. 

We love writing this blog, and plan to grow it even more over the coming months. If this is your first time, welcome! Otherwise, thank you for your support and attention. There's a lot to read on the 'net, and we're thrilled you chose this.

Measuring value

Often in upstream oil and gas we are challenged with a simple question: what's the value? What's the value of that 3D seismic? What's the NPV of this study? How does your professional network affect our bottom line?

Sometimes, at least for me, the first reaction is indignation. Let's take the seismic example; it goes like this:

Finance guy - So, this $28M... 3D seismic. What's the value of this data set?
Geoscientist - What's the value? Of that 3D? That state-of-the-art, high-fold, wide-azimuth, long-offset, high-bandwidth, eco-friendly, ultra-safe 3D seismic survey I just spent four months designing and soliciting bids on?
Finance guy - Yeah
Geoscientist - We have four wells on twenty square kilometres of land. We want to drill forty more. The wells cost $10M each. The seismic will allow us to pick the best locations. It's 3D seismic, the best quality. We always do it. Everyone does it. We can't do subsurface science without it, not very well anyway.
Finance guy - Yeah... Sorry, what's the value of the seismic? Dollars will do.

Finance guy just wants a number. The value is clear to everyone involved. But maybe money is tight this year and finance would like to defer some costs to next year. Maybe we can lower the cost by making the survey smaller, or reducing the fold. Before too long someone utters the unspeakable: 'Value of Information'. The next month of your life becomes a frustrating spreadsheet nightmare of trying to get the process to yield the answer your team wants so you can get on with finding oil and gas.

I think there is a better way. What do you think?

Tell more about me

I thought a neat way to highlight my experience and career interests would be to make up a word cloud from the text in my resume. It's a map of the significant words describing my professional life. Try it out on yourself, and who knows, you might learn to look at yourself in a whole new way :)

When I look at this map, I wish a few words were bigger: business, value-adding, variety. However, I think they are growing!  Developed, is past tense.  I wish to continue to develop and innovate more and technologies and resources.  Problem-Solving; I feel like that is under-represented. Seismic and scholarship seem to be disporportionately large, but I guess I will have to grow other words to catch up with them. This word cloud has omitted places where I have done geoscience; Offshore Australia, Central Alberta, Athabasca Oil Sands, Finland, North Sea, Atlantic Canada.  Words missing altogether; enthusiastic, creative, resourceful, a little geeky?

Setting up shop

My family and I finally moved into our new old house on Sunday. We are slowly gaining the upper hand in the battle of the boxes. The kids have got some of their toys back and are finding their way around the place. I can find most things by looking in only about three cupboards. 

And my new office is almost finished. The last occupant of this place was a boat-builder, and he had a great workshop in the back yard (see pic). I would love to have kept it as-is, but a couple of days of trying to work in the house made it clear that I need a workspace outside the house. 

So I've partitioned off half the space, added a long landscape window, and a floor, and a built-in desk, and it should be comfortable enough to work in for a few months at least, maybe even years.

Even starting with some constraints, it was surprisingly hard to put my finger on exactly what would make my ideal workspace. Here are some of the things I decided I wanted, in no particular order:

• Some sort of view
• Natural light, ideally from a north-facing skylight
• A long desk with no cable clutter
• Plenty of space to tidy hard drives, printers, routers, etc.
• Plenty of space for books and papers
• Somewhere comfortable to sit and read
• Somewhere to work on non-computery stuff (like another desk)
• A wood-burning stove
• A kitchenette and a toilet

The skylight, stove, and sanitation will have to wait, but I think I got most of the rest checked. I'll post a follow-up when it's nearing completion.

Update on 2010-12-30 14:11 by Matt Hall

The office is finished, more or less, and I have moved in, more or less... still haven't figured out lighting and heating, but am working effectively. More or less.