Expert culture is bad for you

Expert culture is bad for you. Not experts themselves, though I prefer not to use the word at all, but a culture that elevates them unduly. I don't like the word because it is usually used to mean something like master, chief, authority, or worst of all, judge. 

What's wrong with expert culture? Lots:

• It disenfranchises everyone else. Non-experts think there are some opinions they are not entitled to. In a highly creative, subjective discipline like ours, this is A Bad Thing.
• This forces them to wait around till the expert can tell them what to do. Which slows everything down. If they have to wait too long, or can't get the expert's attention, or the expert can't or won't get involved, the opportunity, whatever it was, may disappear. 
• Meanwhile, experts are burdened with impossibly high expectations — of always being right or at least deeply insightful. This makes them cautious. So if they're uncertain or uncomfortable, they hang back because there's no upside to being wrong in the expert culture.
• Expert culture encourages knowledge hoarding, because it explicitly connects personal knowledge with glory, and downplays what the rest of the organization knows. The ignorance of the masses highlights the expert's prestige.
• Experts, frustrated with having to tell people what to do all the time, write best practice documents and other edicts, which try to make tricky workflows idiot-proof. But idiot-proof means idiot-friendly — who did you hire?

How to fix it

Better is a culture of expertise. The basic premise is that expertise is everywhere in your organization. You do not, and can not, know where it is. Indeed, its whereabouts will often surprise you. Turns out you hired awesome people after all — and they know stuff. Yay!

In the culture of expertise, what are these people we often call experts? They are still highly experienced people, with unusually broad or deep careers, with profound intelligence or intuition. But now they are free to apply their insight and judgment in more creative and more daring ways — even to things they aren't considered experts in. And their role in this new culture shifts slightly: it becomes the seeking, assessing, parsing, synthesizing, and spreading of expertise in the organization — wherever it is. They become curators, mentors, and champions of excellence. And they will revel in it.

The best experts do this already. How many do you know? Will you step up?

The developer's mind

Humbled by the aura of the legendary Cavendish Labs sitting in the adjacent building next door, I refrain from expressing the full extent of my awe and reverence for this special place. "Sure", I think, "it's no big deal. Let's get on with it". I came to Cambridge to collaborate with Pietro Berkes. He's building Canopy Geo at Enthought. We spent the day spiking, apparently. Working shoulder to shoulder with Pietro was nearly as responsive as dictating a vision to a painter and watching it emerge before my eyes. He's darn good. During my visit, I took notice of some characteristics and guiding principles that top developers, such as himself and his colleagues, bring to their work.

On whiteboarding

The best way to be understood, to connect, or to teach, is to do it one-on-one in front of a whiteboard. It is fitting that all of the walls of their office space are whiteboard walls. Old marks wiped clean but still visible show remnant algorithms sketched out and stacked up upon each other. A well-worn workshop, where writing on the walls is the cultural norm. And for electronic communication? Some are deliberate to only check emails three times a day: first thing in the morning, midday, and mid afternoon. Any more often, would be disruptive to their flow. Email is the enemy of real work, but instant messaging can be be a good productivity tool. 

On discipline

To build something that is extensible takes a good deal of thoughfulness and discipline. Code will survive long after the project is over and the programmer has moved on. This doesn't just mean leaving an adequate documentation trail behind you, but also building a solid foundation that others can contribute to. Being Agile, it turns out, although not the only choice, also takes discipline and diligence in order to be effective. 

On ownership and responsibilty 

Authority is not given, responsibility is taken. Many of the best developers define themselves by the authorship of code and libraries. So attribution is not only necessary politeness, it is a direct line of communication. What body of work would you stand up and speak for? Someone may find a bug at 9:00 am in a different time zone. Will they wait till 2:00 pm to hear from you? Somehow, this decentralized system of self-appointed responsibilty just works. The longevity of emotional and intellectual labour, particuarly in an open source setting, is a fascinating concept. The work becomes more relevant because the developer never stops caring for it. You can change projects, you can change languages, you can change companies, but your work never leaves you. If that notion excites you, you are making an impact. 

The developer knows that prowess is earned by execution. They thrive in an accepted sub-culture of meritocracy: largely free of politics, organizational hierarchies, and other social drama that get in the way of the real work. With a mind cleared to deal with essential tasks, what emerges is the ego of an artist and a creator with the potential to act on it. "Now that we can build anything, what do we do next?"  

EAGE 2013 in a nutshell

I left London last night for Cambridge. On the way, I had a chance to reflect on the conference. The positive, friendly vibe, and the extremely well-run venue. Wi-Fi everywhere, espresso machines and baristas keeping me happy and caffeinated.

Knowledge for sale

I saw no explicit mention of knowledge sharing per se, but many companies are talking about commoditizing or productizing knowledge in some way. Perhaps the most noteworthy was an update from Martyn Millwood Hargrave at Ikon's booth. In addition to the usual multi-client reports, PowerPoint files, or poorly architected database, I think service companies are still struggling to find a model where expertise and insight can be included as a service, or at least a value-add. It's definitely on the radar, but I don't think anyone has it figured out just yet.

Better than swag

Yesterday I pondered the unremarkability of carrot-and-ginger juice and Formula One pit crews. Paradigm at least brought technology to the party. Forget Google Glass, here's some augmented geoscience reality:

Trends good and bad

This notion of 3D seismic vizualization and interpretation is finally coming to gathers. The message: if you are not going pre-stack, you are missing out. Pre-stack panels are being boasted in software demos by the likes of DUG, Headwave, Transform, and more. Seems like this trend has been moving in slow motion for about a decade.

Another bandwagon is modeling while you interpret. I see this as an unfeasible and potentially dangerous claim, but some technologies companies are creating tools and workflows to fast-track the seismic interpretation to geologic model building workflows. Such efficiencies may have a market, but may push hasty solutions down the value chain. 

What do you think? What trends do you detect in the subsurface technology space? 

Where have all the geologists gone?

Fresh off the plane from my vacation in Europe, I spent today exploring the Exhibition at ExCel London, at the 2013 EAGE convention. It's a massive venue, and I spent the entire day there having conversations. I didn't look upon a single PowerPoint slide all day, and it was awesome. 

Seismic domination

This is my first time at the EAGE conference, and I was expecting to see an fairly equal spread of geoscientists and engineers. I was wrong. The exhibition hall at least, is dominated by seismic acquisition and processing companies. Which suggests one thing. There is big business in seismic methods — manufacturing equipment, designing and acquiring surveys, and processing all that data.

Additionally, I counted 17 operating companies out on display. Recruiting and networking hoopla at its finest. In contrast, I don't think I saw one operator on the exhibition floor one month ago at the Calgary GeoConvention.  

Apparently, geophysics is hot. But where have all the geologists gone? Are they lurking in the shadows? Based on the technology represented at the exhibition, are we at the risk of homogenizing the industry and all calling ourselves geophysicists one day?

Even though the diversity of disciplines appears to be lacking, marketing creativity certainly is not. Today, I listened to a string quartet perform at one booth, while sipping on a carrot-and-ginger juice freshly squeezed at another. Moments later, I struggled through a hard-to-hear conversation because the noise from a Formula One pit crew demonstration was deafening. It is both amazing and disturbing the expense companies will rack up to try to be remarkable. But such remarks of the fleeting kind. It fades as quickly as the song, drink, or tire-change is over.

And being a spectator of it all, I am reminded that the remarkability I am after is of the more enduring kind.

Capturing conferences

Yesterday I grumbled about secret meetings. Enough whining, what's the opportunity?

Some technical societies already understand the need for recording proceedings. SPE workshops have clearly-stated deliverables, for example at the Marginal Fields workshop in Cairo later this year:

SPE workshop deliverables

That's more like it. It would be better to publish the proceedings to the world, not just attendees, and to use an open license, but it's definitely a good start.

As we reported earlier, we used a variety of methods to capture the unsession we hosted at the Canada GeoConvention in May — video, photos, drawings and notes, interviews, a wiki, and an article. This is messy and chaotic, but it's also transparent and open, making it easier to reference and more likely that someone (including us!) can use it.

What other things should we be considering? Here are some ideas:

• Livestreaming. This enables people who couldn't make it to take part in at least one or two sessions via streaming video and social media. It's amazingly effective and can increase the audience by a factor of 5 or more. Even better: at the end of it, you have video of everything!
• Published proceedings. This is something we used to do all the time in geoscience, but it takes a lot of coordination. The GSL is the only body I know of that still manages it regularly. Perhaps a wiki-based publishing approach would be easier to arrange?
• Graphic recording. I have witnessed this a few times, and even tried it myself. In a workshop, it's a terrific way to engage the audience in real time; in a conference, it makes for some great conversation pieces in the break. It's also a brilliant way to listen.
• Podcasting. Having a small team of reporters capture the proceedings in short interviews, video clips, and opinion pieces could be a fun way to engage non-attendees, and leave the event with a record of what went on.
• Code and data. We could experiment with more 'doing' sessions, where code is written, wiki pages are hacked on, data is collected, and so on. The product then is clear: a new code repository, open dataset, or set of wiki pages. This one could be the easiest one to pull off, and the most valuable to the community.

Here's more inspiration: the EGU tweeting its round-up from Vienna this morning:

Looking back at the #EGU2013 - a whole lot of online action, excellent scientists, geo-photos, journals & more ow.ly/lE0jH#GeoQ — EGU (@EuroGeosciences) June 6, 2013

Have you seen unusually effective or innovative ways to record events you've been at? What would you like to see? What would you be prepared to do?

---

Update on 2013-06-07 21:51 by Matt Hall

In my inbox today: SEG are offering video of something like 200 talks (of the 1000+) from last year's Annual Meeting. Unfortunately, they're trying to monetize, but at least they're only $0.99. I enjoyed this one on 3D image segmentation live — but haven't tried it online.

The forum that never happened

Recently, I keep seeing this on SEG meeting information:

Note: The mechanical recording of any portion of the [meeting] in any form (photographic, electronic, etc.) is strictly prohibited. Printed reference to the [...] presentations or discussions is not permitted without the consent of the parties involved. All participants are requested to omit public reference to the [...] proceedings in any published work or oral presentation. Only registrants are permitted to attend Forum sessions. Each participant agrees to these regulations when application is accepted [...]

Interesting! 'Regulations' about what a person can and can't say or write about a scientific meeting, a sort of gag order. It goes further: in an attempt to limit what is revealed to the outside world, abstracts are not required at some meetings, only short descriptions. There shall be no evidence of the talk even taking place.

I am convinced that meetings like this are unhelpful, unscientific, and unproductive. Here's why:

We are free. I am a professional scientist and I own my brain. If I want to talk about my field with other professionals, in public if I so wish, then I am entitled to do that. Of course I won't disclose proprietary information, but that's different — proprietary information hasn't been presented at a conference of my competitors.

Public is public. Here's how a forum should work: people should present things they wish to make public. When they're public, they're public, end of story. Asking for secret meetings is like asking for privacy on Facebook. If you want secret, pick up the phone or huddle in dark corners. Or consider the Chatham House Rule.

Secrecy is a bug, not a feature. SEG is a technical society for the advancement of applied geophysics and those who practise it. It's not The Magic Circle. The difference between science and magic is that in science, we do things transparently whenever we can. I know industry is a bit different, but in the interests of innovation and excellence, we need more openness, not less.

No product? No point. If you organize a workshop and there is no tangible outcome — no abstracts, no proceedings (remember those?), no recording — then it's my conviction that there was no point in your workshop, except perhaps for the handful of people who came.

Down with elitism. Surely SEG stands for technical excellence among all of its members, not just the privileged few with the time and resources to write papers and travel to workshops? If you're using the resources of technical societies (their time, attention, and marketing clout) then I believe it's your duty to the membership, and to the science as a whole, to share.

An unsolved problem. Corporate secrecy was identified as one of the top unsolved problems of subsurface science in our recent unsession. So what are we playing at? Are we professionals and scientists or just industrial magicians, selfishly hoarding our ideas and data, and slowing innovation down for everyone? What do you think?

Update on 2013-06-06 12:21 by Matt Hall

Don't miss the follow-up article — Capturing conferences — in which I stop whining and propose some things I think we should be doing. Let us know what you think! 

May linkfest

The monthly News post wasn't one of our most popular features, so it's on the shelf. Instead, I thought I'd share all the most interesting, quirky, mind-blowing, or just plain cool things I've spotted on the web over the last month.

– Do not miss this. One of them stands out above all the others. If you like modern analogs and satellite imagery, you're going to love Google Earth Engine. I've started a list of geologically interesting places to visit — please add to it!

– More amazing images. I'll never get bored of looking at gigapans, and Callan Bentley's are among the best geological ones. I especially like his annotated ones.

– Classic blog. Greg Gbur writes one of the best physics blogs, and his focus is on optics, so there's often good stuff there for geophysicists. This post on Chladni patterns is pure acoustic goodness and well worth a slow read. 

– New geoscience blog. Darren Wilkinson is a young geoscientist in the UK, and writes a nice geeky blog about his research. 

– Brilliant and simple. Rowan Cockett is a student at UBC, but builds brilliant geological web apps on the side. He has a knack for simplicity and his latest creation makes stereonets seem, well, simple. Impressive. 

– New magazine. Kind of. There's not enough satire or ragging in the petroleum press, so it's refreshing to hear of Proved Plus Probable, a fairly wacky weekly online rag emanating from Calgary [thanks to Dan for the tip!]. Top headline: Legendary geologist invents new crayons. 

– Counter-factual geology. I love these pictures of an imagined ring around earth.

– Never buy graph paper again. Make some just how you like it!

– Bacon. It was a revelation to find that some rocks look just like bacon.

That's it! I share most of this sort of thing on Twitter. Really useful stuff I tend to stick on my pinboard — you're welcome to browse. If you have a geological or geeky bookmark collection, feel free to share it in the comments!

The deliberate search for innovation & excellence

Collaboration, knowledge sharing, and creativity — the soft skills — aren't important as ends in themselves. They're really about getting better at two things: excellence (your craft today) and innovation (your craft tomorrow). Soft skills matter not because they are means to those important ends, but because they are the only means to those ends. So it's worth getting better at them. Much better.

One small experiment

The Unsession three weeks ago was one small but deliberate experiment in our technical community's search for excellence and innovation. The idea was to get people out of one comfort zone — sitting in the dark sipping coffee and listening to a talk — and into another — animated discussion with a roomful of other subsurface enthusiasts. It worked: there was palpable energy in the room. People were talking and scribbling and arguing about geoscience. It was awesome. You should have been there. If you weren't, you can get a 3-minute hint of what you missed from the feature film...

Go on, share the movie — we want people to see what a great time we had! 

Big thank you to the award-winning Craig Hall Video & Photography (no relation :) of Canmore, Alberta, for putting this video together so professionally. Time lapse, smooth pans, talking heads, it has everything. We really loved working with them. Follow them on Twitter. 

Proceedings of an unsession

Two weeks ago today Evan and I hosted a different kind of session at the Canada GeoConvention. It was an experiment in collaboration and integration, and I'm happy to say it exceeded our expectations. We will definitely be doing it again, so if you were there, or even if you weren't, any and all feedback will help ensure the dial goes to 11.

One of the things we wanted from the session was evidence. Evidence of conversation, innovation, and creative thinking. We took home a great roll of paper and sticky notes, and have now captured it all in SubSurfWiki, along with notes from the event. You are invited to read and edit. Be bold! And please share the link...

  ageo.co/unsession

The video from the morning is in the editing suite right now: watch for that too.

We have started a write-up of the morning. If you came to the session, please consider yourself a co-author: your input and comments are welcome. You might be unaccustomed to editing a community document, but don't be shy — that's what it's there for. 

We want to share two aspects of the event on the blog. First, the planning and logistics of the session — a cheatsheet for when we (or you!) would like to repeat the experience. Second, the outcomes and insights from it — the actual content. Next time: planning an unsession.

Fitting a model to data

In studying the earth, we can't afford to take enough observations, and they will never be free of noise. So if you say you do geoscience, I hereby challenge you to formulate your work as a mathematical inverse problem. Inversion is a question: given the data, the physical equations, and details of the experiment, what is the distribution of physical properties? To answer this question we must address three more fundamental ones (Scales, Smith, and Treitel, 2001):

• How accurate is the data? Or what does fit mean?
• How accurately can we model the response of the system? Have we included all the physics that can contribute signifcantly to the data?
• What is known about the system independent of the data? There must be a systematic procedure for rejecting unreasonable models that fit the data as well.

Setting up an inverse problem means coming up with the equations that contain the physics and geometry of the system under study. The method for solving it depends on the nature of the system of equations. The simplest is the minimum norm solution, and you've heard of it before, but perhaps under a different name.

To fit is to optimize a system of equations

For problems where the number of observations is greater than the number of unknowns, we want to find which unknowns fit the best. One case you're already familiar with is the method of least squares — you've used it fitting a line of through a set of points. A line is unambiguously described by only two parameters: slope a and y-axis intercept b. These are the unknowns in the problem, they are the model m that we wish to solve for. The problem of line-fitting through a set of points can be written out like this,

As I described in a previous post, the system of the problem takes the form d = Gm, where each row links a data point to an equation of a line. The model vector m (M × 1), is smaller than the data d (N × 1) which makes it an over-determined problem, and G is a N × M matrix holding the equations of the system.

Why cast a system of equations in this matrix form? Well, it turns out that the the best-fit line is precisely,

which are trivial matrix operations, once you've written out G.  T means to take the transpose, and –1 means the inverse, the rest is matrix multiplication. Another name for this is the minimum norm solution, because it defines the model parameters (slope and intercept) for which the lengths (vector norm) between the data and the model are a minimum. 

One benefit that comes from estimating a best-fit model is that you get the goodness-of-fit for free. Which is convenient because making sense of the earth doesn't just mean coming up with models, but also expressing their uncertainty, in terms of the errors with which they are linked.

I submit to you that every problem in geology can be formulated as a mathematical inverse problem. The benefit of doing so is not just to do math for math's sake, but it is only through quantitatively portraying ambiguous inferences and parameterizing non-uniqueness that we can do better than interpreting or guessing. 

Reference (well worth reading!)

Scales, JA, Smith, ML, and Treitel, S (2001). Introductory Geophysical Inverse Theory. Golden, Colorado: Samizdat Press.