Take one, make one

There’s a teaching method originating in medicine known as “see one, do one, teach one”. I like it because it underscores hands-on practice and knowledge sharing as essential steps in developing a craft — and it works. Today, I want to urge you to take a challenge, then make one for others.

First, what’s the challenge?

A couple of years ago, inspired by the annual Advent of Code challenges, we introduced the kata, a set of coding challenges especially for geoscientists. For a long time we sent them to students in our Geocomputing class, to encourage them to keep coding. Now we just tell everyone about them.

At the time we announced the kata, there were five puzzles. Today, there are 11: four beginner-friendly challenges, four intermediate ones, and three quite hard ones. Topics range from data munging to map indexing, and from digital elevation models to fractures.

💡 If you want to try one, this Colab is the easiest way to get started: https://ageo.co/kata-live

Now make one!

Once you’ve got an idea of how these things work, you might want to try your hand at making one. Once you have an idea for a short task, you need a way to generate a random dataset. For example, for the sample-names challenge, I have a function that generates a random set of sample names, composed of several parts (a number, a basin, a formation, a data, etc).

When you have a dataset, you can ask some questions about it. Start with an one, and build from there. The last question (there can be 3 or 4), should be a somewhat realistic challenge for this kind of data. Each question needs a hint, and each question must have only one possible answer (this is the tricky bit!).

If you fancy trying your hand at it, check out our new kata-dev repository on GitHub. There is a demo challenge there, which is also live on the kata server, so you can see how it all works. Good luck!

---

Whether or not you try making a challenge for your peers, so let us know how you get on in the #kata-challenges channel on the Software Underground. We’re always ready to answer questions about them.

Advice for a new hacker

So you’ve signed up for a hackathon — or maybe you’ve seen an event and you’re still thinking about it.

First thing: I can almost guarantee that you will not regret it, so if you haven’t committed yet, I challenge you to go and sign up now.

But even once you’ve chosen to go, maybe you feel nervous about your skills, or are worried about spending two days with strangers, or aren’t sure about the idea of competitive coding. Someone asked me recently how to prepare — technically and mentally — for the event.

I should say that I’ve only participated in a couple of hackathons, so I definitely don’t know everything there is to know. But I have organized more than 20 hackathons, and helped people skill up for them and (I hope!) enjoy them. Here are the top 10-ish things you can to do to get the most out of the event:

1. Brush up on your coding. Before the event, find out a bit about what kinds of projects are in the offing. If it’s a machine learning theme, brush up on your data science. Maybe image processing or text processing will be needed. Data management skills and database manipulation are always appreciated. Familiarty with a cloud environment, e.g. AWS, will help.

2. Find a friend. Either take someone with you, or find a friendly face when you get there. It’s 100% possible to navigate the experience on your own, but much more fun with a partner.

3. Dive in. You get out of the event what you put in. It’s like most learning experiences. You need an open mind, an enthusiastic demeanour, and a can-do attitude.

4. Contribute. There’s never enough time, so you are a much-needed part of your team, but unless there’s a strong effort to coordinate the project, it’ll be a bit unstructured. You’ll have to take the initiative on things.

5. Use a kanban. To help team members see the big picture and select tasks for themselves, put them on stickies on a nearby board. Make 3 areas: ‘to do’, ‘in progress’ and ‘done’. The goal is to move them from left to right.

6. Ask for help. Every event Agile runs has non-hackers around to help out with stuff — anything from dietary needs to datasets to coding advice. Don’t get stuck on something, find someone to help you.

7. Take breaks. You and your team should go for a short walk every 90 minutes or so. Relax a bit, but also get caught up: get progress reports from everyone, re-evaluate the goals, identify issues. You will find more clarity away from your keyboards.

8. Work backwards from the demo. A good strategy is to outline what would make a killer demo of the project you have selected. Include at least one “Wow” feature if at all possible. Then work out what you need to either fake or build to make that demo. Build what you can, fake the rest.

9. Check in with the other teams. This might not fly at highly competitive events, but at more casual affairs or if everyone is working on different projects, try chatting to some other teams, especially during breaks.

10. Label your equipment. Hackathons are pretty chaotic, and although 99.9% of hackers are awesome, it’s still a roomful of strangers, so label the gear you care about. And of course keep your phone and computer locked.

11. Reciprocate. Almost all these bits of advice have corollaries: be friendly and welcoming, accept contributions from others, give help if asked, and so on. Hackathons are social events as much as technical ones — enjoy meeting and collaborating with others.

If you have signed up for an event — I hope you love it! Do let us know how you get along. Or, if you’ve already been to a hackathon and have some advice to share — leave a comment below.

---

If you’re looking for an event to go to and you’re in western Europe — here’s one! It’s the FORCE Machine Learning Hackathon in Stavanger, Norway. I recently wrote about it — check it out.

If you’re looking for subsurface or geoscience project ideas, then I have a lot of reading for you. Check out the long list of hackathons reports on this blog. You can also dive into the Software Underground Slack to discuss project ideas there.

Feel superhuman: learning and teaching geocomputing

Diego teaching in Houston in 2018.

It’s five years since we started teaching Python to geoscientists. To be honest, it might have been premature. At the time, Evan and I were maybe only two years into serious, daily use of Python. But the first class, at the Atlantic Geological Society’s annual meeting in February 2014, was free so the pressure was not too high. And it turns out that only being a step or two ahead of your students can be an advantage. Your ‘expert blind spot’ is partially sighted not completely blind, because you can clearly remember being a noob.

Being a noob is a weird, sometimes very uncomfortable, even scary, feeling for some people. Many of us are used to feeling like experts, at least some of the time. Happy, feeling like a noob is a core competency in programming. Learning new things is a more or less hourly experience for coders. Even a mature language like Python evolves fast enough that it’s hard to keep up. Instead of feeling threatened or exhausted by this, I think the best strategy is to enjoy it. You’ll never be done, there are (way) more questions than answers, and you can learn forever!

One of the bootcamp groups at the Copenhagen hackathon in 2018

This week we’re teaching our 40th course. Last year alone we gave digital superpowers to 325 people, mostly geoscientists, Not all of them learned to code, as such — some people already could, and some found out theydidn’t like it… coding really isn’t for everyone. But I think all of them learned something new about technology, and how it can serve them and their science. I hope all of them look at spreadsheets, and Petrel, and websites differently now. I think most of them want, at some point, to learn more. And everyone is excited about machine learning.

The expanding community of quantitative earth scientists

This year we’ve already spent 50 days teaching, and taught 174 people. Imagine that! I get emotional when I think about what these hundreds of new digital geoscientists and engineers will go and do with their new skills. I get really excited when I see what they are already doing — when they come to hackathons, send us screenshots, or write papers with beautiful figures. If the joy of sharing code and collaborating with peers has also rubbed off on them, there’s no telling where it could lead.

Matt teaching in Aberdeen in October 2018

The last nine months or so have been an adventure. Teaching is not supposed to be what Agile is about. We’re a consulting company, a technology company. But for now we’re mostly a training company — it’s where we’re needed. And it makes sense... Programming is fundamentally about knowledge sharing. Teaching is about helping, collaborating. It’s perfect for us.

Besides, it’s a privilege and a thrill to meet all these fantastically smart, motivated people and to hear about their projects and their plans. Sometimes I wish it didn’t mean leaving my family in Nova Scotia and flying to Houston and London and Kuala Lumpur and Kalamazoo… but mostly I wish we could do more of it. Especially when we get comments like these:

“Given how ‘dry’ programming can be, it was DYNAMIC.”
”Excellent teachers with geoscience background.”
”Great instructors, so so approachable, even for newbies like me.”
”Great course [...] Made me realize what could be done in a short time.”
”My only regret was not taking a class like this sooner.”
”Very positive, feel superhuman.”

How many times have you felt superhuman at work recently?

The courses we teach are evolving and expanding in scope. But they all come back to the same thing: growing digital skills in our profession. This is critical because using computers for earth science is really hard. Why? The earth is weird. We’ve spent hundreds of years honing conceptual models, understanding deep time, and figuring out complex spatial relationships.

If data science eats the subsurface without us, we’re all going to get indigestion. Society needs to better understand the earth — for all sorts of reasons — and it’s our duty to build and adopt the most powerful analytical tools available so that we can help.

---

Learning resources

If you can’t wait to get started, here are some suggestions:

• learnpython.org is a really nice resource for beginners, with interactive tutorials — and not just in Python!

• I took the Udacity CS 101 class with Dave Evans in 2012, and I loved it. I’ve since done other Udacity courses.

• If you’re the book type, Learning With Python by Allen P. Downey is supposed to be great, and it’s free.

• Our X lines of Python blog series has several beginner-friendly lessons. My favourite is the one about wedges. You should also check out the Geophysical Tutorials series in The Leading Edge.

• You might like to get on the public, free Slack Software Underground — there are lots of geo-Python folks there.

Classroom courses are a big investment in dollars and time, but they can get you a long way really quickly. Our courses are built especially for subsurface scientists and engineers. As far as I know, they are the only ones of their kind. If you think you’d like to take one, talk to us, or look out for a public course. You can find out more or sign up for email alerts here >> https://agilescientific.com/training/

Last thing: I suggest avoiding DataCamp, because of sexual misconduct by an executive, compounded by total inaction, dishonest obfuscation, and basically failing spectacularly. Even their own trainers have boycotted them. Steer clear.

Closing the analytics–domain gap

I recently figured out where Agile lives. Or at least where we strive to live. We live on the isthmus — the thin sliver of land — between the world of data science and the domain of the subsurface.

We’re not alone. A growing number of others live there with us. There’s an encampment; I wrote about it earlier this week.

Backman’s Island, one of my favourite kayaking destinations, is a passable metaphor for the relationship between machine learning and our scientific domain.

Closing the gap in your organization

In some organizations, there is barely a connection. Maybe a few rocks at low tide, so you can hop from one to the other. But when we look more closely we find that the mysterious and/or glamorous data science team, and the stories that come out of it, seem distinctly at odds with the daily reality of the subsurface professionals. The VP talks about a data-driven business, deep learning, and 98% accuracy (whatever that means), while the geoscientists and engineers battle with raster logs, giant spreadsheets, and trying to get their data from Petrel into ArcGIS (or, help us all, PowerPoint) so they can just get on with their day.

We’re not going to learn anything from those organizations, except maybe marketing skills.

We can learn, however, from the handful of organizations, or parts of them, that are serious about not only closing the gap, but building new paths, and infrastructure, and new communities out there in the middle. If you’re in a big company, they almost certainly exist somewhere in the building — probably keeping their heads down because they are so productive and don’t want anyone messing with what they’ve achieved.

Here are some of the things they are doing:

• Blending data science teams into asset teams, sitting machine learning specialists with subsurface scientists and engineers. Don’t make the same mistake with machine learning that our industry made with innovation — giving it to a VP and trying to bottle it. Instead, treat it like Marmite: spread it very thinly on everything.*

• Treating software like knowledge sharing. Code is, hands down, the best way to share knowledge: it’s unambiguous, tested (we hope anyway), and — above all — you can actually use it. Best practice documents are I’m afraid, not worth the paper they would be printed on if anyone even knew how to find them.

• Learning to code. OK, I’m biased because we train people… but it seriously works. When you have 300 geoscientists in your organization that embrace computational thinking, that can write a function in Python, that know what a support vector machine is for — that changes things. It changes every conversation.

• Providing infrastructure for digital science. Once you have people with skills, you need people with powers. The power to install software, instantiate a virtual machine, or recruit a coder. You need people with tools, like version control, continuous integration, and communities of practice.

• Realizing that they need to look in new places. Those much-hyped conversations everyone is having with Google or Amazon are, admittedly, pretty cool to see in the extractive industries (though if you really want to live on the cutting edge of geospatial analytics, you should probably be talking to Uber). You will find more hope and joy in Kaggle, Stack Overflow, and any given hackathon than you will in any of the places you’ve been looking for ‘innovation’ for the last 20 years.

This machine learning bandwagon we’re on is not about being cool, or giving keynotes, or saying ‘deep learning’ and ‘we’re working with Google’ all the time. It’s about equipping subsurface professionals to make better and safer scientific, industrial, and business decisions with more evidence and more certainty.

And that means getting serious about closing that gap.

---

I thought about this gap, and Agile’s place in it — along with the several hundred other digital subsurface scientists in the world — after drawing an attempt at drawing the ‘big picture’ of data science on one of our courses recently. Here’s a rendering of that drawing, without further comment. It didn’t quite fit with my ‘sliver of land’ analogy somehow…

On the left, the world of ‘advanced analytics’, on the right, how the disciplines of data science and earth science overlap on and intersect the computational world. We live in the green belt. (yes, we could argue for hours about these terms, but let’s not.)

---

* If you don’t know what Marmite is, it’s not too late to catch up.

Digitalization... of what?

I've been hearing a lot about 'digitalization', or 'digital transformation', recently. What is this buzzword?

The general idea seems to be: exploit lots and lots of data (which we already have), with analytics and machine learning probably, to do a better job finding and producing fuel and energy safely and responsibly.

At the centre of it all is usually data. Lots of data, usually in a lake. And this is where it all goes wrong. Digitalization is not about data. And it's not about technology either. Or cloud. Or IoT.

Interest in the terms "digital transformation" and "digitalization" since 2004, according to Google Trends. The data reveal a slight preference for the term "digitalization" in central and northern Europe. Google Ngram Viewer indicates that the term "digitalization" has been around for over 100 years, but it is also a medical term connected with the therapeutic use of digitalis. Just to be clear, that's not what we're talking about.

It's about people

Oh no, here I go with the hand-wavy, apple-pie "people not process" nonsense... well, yes. I'm convinced that it's humans we're transforming, not data or technology. Or clouds. Or Things.

I think it's worth spelling out, because I think most corporations have not grasped the human aspect yet. And I don't think it's unreasonable to say that petroleum has a track record of not putting people at the centre of its activities, so I worry that this will happen again. This would be bad, because it might mean that digitalization not only fails to get traction — which would be bad enough because this revolution is long overdue — but also that it causes unintended problems.

Without people, digital transformation is just another top-down 'push' effort, with too much emphasis on supply. I think it's smarter to create demand, or 'pull', so that professionals are asking for support, and tools, and databases, and are engaged in how those things are created.

Put technical professionals at the heart of the revolution, and the rest will follow. The inverse is not true.

Strategies

This is far from an exhaustive list, but here are some ideas for ways to get ahead in digital transformation:

• Make it easy for digitally curious people to dip a toe in. Build a beginner-friendly computing environment, and encourage people to use it. Challenge your IT people to support a culture of experimentation and creativity. 
• Give those curious professionals access to professional development channels, whether it's our courses, other courses, online channels like Lynda.com or Coursera, or whatever. 
• Build a community of practice for 'scientific computing'. Whether it's a Yammer group or something more formal, be sure to encourage frequent face-to-face meetups, and perhaps an intranet portal.
• Start to connect subsurface professionals with software engineers, especially web programmers and data scientists, elsewhere in the organization. I think the best way is to embed programmers into technical teams. 
• Encourage participation in external channels like conferences and publications, data science contests, hackathons, open source projects, and so on. I guarantee you'll see a step change in skills and enthusiasm.

The bottom line is that we're looking for a profound culutral change. It will be slow. More than that, it needs to be slow. It might only take a year or two to get traction for an idea like "digital first". But deeper concepts, like "machine readable microservices" or "data-driven decisions" or "reproducible workflows", must take longer because you can't build that high without a solid foundation. Successfully applying specific technologies like deep learning, augmented reality, or blockchain, will certainly require a high level of technology literacy, and will take years to get right.

What's going on with scientific computing in your organization? Are you 'digitally curious'? Do you feel well-supported? Do you know others in your organization like you?

---

The circuit board image in the thumbnail for this post is by Carl Drougge, licensed CC-BY-SA.

x lines of Python: web scraping and web APIs

The Web is obviously an incredible source of information, and sometimes we'd like access to that information from within our code. Indeed, if the information keeps changing — like the price of natural gas, say — then we really have no alternative.

Fortunately, Python provides tools to make it easy to access the web from within a program. In this installment of x lines of Python, I look at getting information from Wikipedia and requesting natural gas prices from Yahoo Finance. All that in 10 lines of Python — total.

As before, there's a completely interactive, live notebook version of this post for you to run, right in your browser. Quick tip: Just keep hitting Shift+Enter to run the cells. There's also a static repo if you want to run it locally.

Geological ages from Wikipedia

Instead of writing the sentences that describe the code, I'll just show you the code. Here's how we can get the duration of the Jurassic period fresh from Wikipedia:

url = "http://en.wikipedia.org/wiki/Jurassic"
r = requests.get(url).text
start, end = re.search(r'<i>([\.0-9]+)–([\.0-9]+)&#160;million</i>', r.text).groups()
duration = float(start) - float(end)
print("According to Wikipedia, the Jurassic lasted {:.2f} Ma.".format(duration))

The output:

According to Wikipedia, the Jurassic lasted 56.30 Ma.

There's the opportunity for you to try writing a little function to get the age of any period from Wikipedia. I've given you a spot of help, and you can even complete it right in your browser — just click here to launch your own copy of the notebook.

Gas price from Yahoo Finance

url = "http://download.finance.yahoo.com/d/quotes.csv"
params = {'s': 'HHG17.NYM', 'f': 'l1'}
r = requests.get(url, params=params)
price = float(r.text)
print("Henry Hub price for Feb 2017: ${:.2f}".format(price))

Again, the output is fast, and pleasingly up-to-the-minute:

Henry Hub price for Feb 2017: $2.86

I've added another little challenge in the notebook. Give it a try... maybe you can even adapt it to find other live financial information, such as stock prices or interest rates.

What would you like to see in x lines of Python? Requests welcome!

Working without a job

I have drafted variants of this post lots of times. I've never published them because advice always feels... presumptuous. So let me say: I don't have any answers. But I do know that the usual way of 'finding work' doesn't work any more, so maybe the need for ideas, or just hope, has grown. 

Lots of people are out of work right now. I just read that 120,000 jobs have been lost in the oil industry in the UK alone. It's about the same order of magnitude in Canada, maybe as much as 200,000. Indeed, several of my friends — smart, uber-capable professionals — are newly out of jobs. There's no fat left to trim in operator or service companies... but the cuts continue. It's awful.

The good news is that I think we can leave this downturn with a new, and much better, template for employment. The idea is to be more resilient for 'next time' (the coming mergers, the next downturn, the death throes of the industry, that sort of thing).

The tragedy of the corporate professional 

At least 15 years ago, probably during a downturn, our corporate employers started telling us that we are responsible for our own careers. This might sound like a cop-out, maybe it was even meant as one, but really it's not. Taken at face value, it's a clear empowerment.

My perception is that most professionals did not rise to the challenge, however. I still hear, literally all the time, that people can't submit a paper to a conference, or give a talk, or write a blog, or that they can't take a course, or travel to a workshop. Most of the time this comes from people who have not even asked, they just assume the answer will be No. I worry that they have completely given in; their professional growth curtailed by the real or imagined conditions of their employment.

More than just their missed opportunity, I think this is a tragedy for all of us. Their expertise effectively gone from the profession, these lost scientists are unknown outside their organizations.

Many organizations are happy for things to work out that way, but when they make the situation crystal clear by letting people go, the inequity is obvious. The professional realizes, too late, that the career they were supposed to be managing (and perhaps thought they were managing by completing their annual review forms on time) was just that — a career, not a job. A career spanning multiple jobs and, it turns out, multiple organizations.

I read on LinkedIn recently someone wishing recently let-go people good luck, hoping that they could soon 'resume their careers'. I understand the sentiment, but I don't see it the same way. You don't stop being a professional, it's not a job. Your career continues, it's just going in a different direction. It's definitely not 'on hold'. If you treat it that way, you're missing an opportunity, perhaps the best one of your career so far.

What you can do

Oh great, unsolicited advice from someone who has no idea what you're going through. I know. But hey, you're reading a blog, what did you expect? 

• Do you want out? If you think you might want to leave the industry and change your career in a profound way, do it. Start doing it right now and don't look back. If your heart's not in this work, the next months and maybe years are really not going to be fun. You're never going to have a better run at something completely different.
• You never stop being a professional, just like a doctor never stops being a doctor. If you're committed to this profession, grasp and believe this idea. Your status as such is unrelated to the job you happen to have or the work you happen to be doing. Regaining ownership of our brains would be the silveriest of linings to this downturn.
• Your purpose as a professional is to offer help and advice, informed by your experience, in and around your field of expertise. This has not changed. There are many, many channels for this purpose. A job is only one. I firmly believe that if you create value for people, you will be valued and — eventually — rewarded.
• Establish a professional identity that exists outside and above your work identity. Get your own business cards. Go to meetings and conferences on your own time. Write papers and articles. Get on social media. Participate in the global community of professional geoscientists. 
• Build self-sufficiency. Invest in a powerful computer and fast Internet. Learn to use QGIS and OpendTect. Embrace open source software and open data. If and when you get some contracting work, use Tick to count hours, Wave for accounting and invoicing, and Todoist to keep track of your tasks. 
• Find a place to work — I highly recommend coworking spaces. There is one near you, I can practically guarantee it. Trust me, it's a much better place to work than home. I can barely begin to describe the uplift, courage, and inspiration you will get from the other entrepreneurs and freelancers in the space.
• Find others like you, even if you can't get to a coworking space, your new peers are out there somewhere. Create the conditions for collaboration. Find people on meetup.com, go along to tech and start-up events at your local university, or if you really can't find anything, organize an event yourself! 
• Note that there are many ways to make a living. Money in exchange for time is one, but it's not a very efficient one. It's just another hokey self-help business book, but reading The 4-Hour Workweek honestly changed the way I look at money, time, and work forever.
• Remember entrepreneurship. If you have an idea for a new product or service, now's your chance. There's a world of making sh*t happen out there — you genuinely do not need to wait for a job. Seek out your local startup scene and get inspired. If you've only ever worked in a corporation, people's audacity will blow you away.

If you are out of a job right now, I'm sorry for your loss. And I'm excited to see what you do next.

The plainest English

If you're not already reading xkcd — the must-read sciencey thrice-weekly comic strip — then please give it a try. It's good for you. Check out this wonderful description of the Saturn V rocket, aka Up Goer Five, using only the 1000 most common words in English →

This particular comic took on a life of its own last week, when Theo Sanderson built a clever online text editor that parses your words and highlights the verboten ones. Then, following the lead of @highlyanne, a hydrologist, scientists all over Twitter quickly started describing and sharing parsimonious descriptions of what they do. Anne and her partner in crime, @Allochthonous, then compiled a log of every description they could find. It's worth looking at, though it would take a while to read them all. 

What's it like using only the simplest words? I tried to define a well...

A deep, round, empty space in the ground that is only about as wide as your hand. The empty space is very deep: up to about seven tens of hundreds of times as deep as a man is tall. It is full of water. After making the empty space, we can lower small computers into it. As we pull them out, the computers tell us things about the rocks they can 'see' — like how fast waves move through them, or how much water the rocks have in them.

It's quite hard. But refreshingly so. Here's reflection seismic...

We make a very loud, short sound on the land or in the water — like a cracking sound. The sound waves go down through the rocks under the ground. As they do so, some of them come back — just as waves come back from the side of a body of water when you throw in a small rock. We can listen to the sound waves that come back, and use a computer to help make a picture of what it looks like under the ground.

Is a world without jargon dumbed down, or opened up? What is it we do again?...

It is very hard to do this work. It takes a lot of money and a long time. The people that do it have to think hard about how to do it without hurting other people or the world we live in. We don't always manage to do it well, but we try to learn from the past so we can do better next time. Most people think we should stop, but if we did, the world would go dark, our homes would be cold (or hot), and people would not be able to go very far.

Check out Up Goer Six — Theo's new editor that colour codes each word according to just how common it is. Try it — what do you do for a living? 

The image is licensed CC-BY-NC-2.5 by Randall Munroe at xkcd.com.

Journalists are scientists

Tim Radford. Image: Stevyn Colgan.On Thursday I visited The Guardian’s beautiful offices in King’s Cross for one of their Masterclass sessions. Many of them have sold out, but Tim Radford’s science writing evening did so in hours, and the hundred-or-so budding writers present were palpably excited to be there. The newspaper is one of the most progressive news outlets in the world, and boasts many venerable alumni (John Maddox and John Durant among them). It was a pleasure just to wander around the building with a glass of wine, with some of London’s most eloquent nerds.

Radford is not a trained scientist, but a pure journalist. He left school at 16, idolized Dylan Thomas, joined a paper, wrote like hell, and sat on almost every desk before mostly retiring from The Guardian in 2005. He has won four awards from the Association of British Science Writers. More people read any one of his science articles on a random Tuesday morning over breakfast than will ever read anything I ever write. Tim Radford is, according to Ed Yong, the Yoda of science writers.

Within about 30 minutes it became clear what it means to be a skilled writer: Radford’s real craft is story-telling. He is completely at home addressing a crowd of scientists — he knows how to hold a mirror up to the geeks and reflect the fun, fascinating, world-changing awesomeness back at them. “It’s a terrible mistake to think that because you know about a subject you are equipped to write about it,” he told us, getting at how hard it is to see something from within. It might be easier to write creatively, and with due wonder, about fields outside our own.

Some in the audience weren’t content with being entertained by Radford, watching him in action as it were, preferring instead to dwell on controversy. He mostly swatted them aside, perfectly pleasantly, but one thing he was having none of was the supposed divide between scientists and journalists. Indeed, Radford asserted that journalists and scientists do basically the same thing: imagine a story (hypothesis), ask questions (do experiments), form a coherent story (theory) from the results, and publish. Journalists are scientists. Kind of.

I loved Radford's committed and unapologetic pragmatism, presumably the result of several decades of deadlines. “You don’t have to be ever so clever, you just have to be ever so quick,” and as a sort of corollary: “You can’t be perfectly right, but you must be mostly right.” One questioner accused journalists of sensationalising science (yawn). “Of course we do!” he said — because he wants his story in the paper, and he wants people to read it. Specifically, he wants people who don’t read science stories to read it. After all, writing for other people is all about giving them a sensation of one kind or another.

I got so much out of the 3 hours I could write at least another 2000 words, but I won’t. The evening was so popular that the paper decided to record the event and experiment with a pay-per-view video, so you can get all the goodness yourself. If you want more Radford wisdom, his Manifesto for the simple scribe is a must-read for anyone who writes.

Tim Radford's most recent book, The Address Book: Our Place in the Scheme of Things, came out in spring 2011.

The photograph of Tim Radford, at The World's Most Improbable Event on 30 September, is copyright of Stevyn Colgan, and used with his gracious permission. You should read his blog, Colganology. The photograph of King's Place, the Guardian's office building, is by flickr user Davide Simonetti, licensed CC-BY-NC.

Update on 2012-10-07 10:12 by Matt Hall

I was surprised to find no Wikipedia article on Tim Radford, so I created one. If you know anything more about Mr Radford, please add to it (with citations, of course!).

I also made an edit to this post: I changed the wording of my reference to Radford as Yoda, to better match the tweet.

Learn to program

This is my contribution to the Accretionary Wedge geoblogfest, number 38: Back to School. You can read all about it, and see the full list of entries, over at Highly Allochthonous. To paraphrase Anne's call to words:

What do you think students should know? What should universities be doing better? What needs do you see for the rising generation of geoscientists? What skills and concepts are essential? How important are things like communication and quantitative skills versus specific knowledge about rocks/water/maps?

Learn to program

The first of doubtless many moments of envy of my kids' experience of childhood came about two years ago when my eldest daughter came home from school and said she'd been programming robots. Programming robots. In kindergarten. 

For the first time in my life, I wished I was five. 

Most people I meet and work with do not know how to make a computer do what they want. Instead, they are at the mercy of the world's programmers and—worse—their IT departments. The accident of the operating system you run, the preferences of those that came before you, and the size of your budget should not determine the analyses and visualizations you can perform on your data. When you read a paper about some interesting new method, imagine being able to pick up a keyboard and just try it, right now... or at least in an hour or two. This is how programmers think: when it comes to computers at least, their world is full of possibility, not bound by software's edges or hardwired defaults.

I want to be plain about this though: I am not suggesting that all scientists should become programmers, hacking out code, testing, debugging, and doing no science. But I am suggesting that all scientists should know how computer programs work, why they work, and how to tinker. Tinkering is an underrated skill. If you can tinker, you can play, you can model, you can prototype and, best of all, you can break things. Breaking things means learning, rebuilding, knowing, and creating. Yes: breaking things is creative.

But there's another advantage to learning to program a computer. Programming is a special kind of problem-solving, and rewards thought and ingenuity with the satisfaction of immediate and tangible results. Getting it right, even just slightly, is profoundly elating. To get these rewards more often, you break problems down, reducing them to soluble fragments. As you get into it, you appreciate the aesthetics of code creation: like equations, computer algorithms can be beautiful.

The good news for me and other non-programmers is that it's never been faster or simpler to give programming a try. There are even some amazing tools to teach children and other novices the concepts of algorithms and procedures; MIT's Scratch project is a leader in that field. Some teaching tools, like the Lego MINDSTORMS robotics systems my daughter uses, and App Inventor for Android (right), are even capable of building robust, semi-scientific applications. 

Chances are good that you don't even need to install anything to get started. If you have a Mac or a Linux machine then you already have instant access to scripting-cum-programming languages like the shell, AWK, Perl and Python. There's even a multi-language interpreter online at codepad.org. These languages are very good places to start: you can solve simple problems with them very quickly and, once you've absorbed the basics, you'll use them every day. Start on AWK now and you'll be done by lunchtime tomorrow. 

For what's it's worth, here are a few tips I'd give anyone learning to program:

• Don't do anything until you have a specific, not-too-hard problem to solve with a computer
• If you can't think of anything, the awesome Project Euler has hundreds of problems to solve
• Choose a high-level language like Python, Perl, or perhaps even Java; stay away from FORTRAN and C
• Buy no more than one single book, preferably a thick one with a friendly title from O'Reilly
• Don't do a course before you've tinkered on your own for a bit, but don't wait too long either (here's one)
• Learn to really use Google: it's the fastest way to figure out what you want to do
• Have fun brushing up on your math, especially trig, time series analysis, and inverse theory
• Share what you build: help others learn and get more open

Bust out of the shackles of other people's software: learn to program!

Update on 2011-10-02 00:38 by Matt Hall

If you have some programming or scripting skillz already, here's another codepad-like to look at. The nicely-designed repl.it can help you explore your multi-lingual side and try out a few languages you might have wondered about. Examples included.