The Data Farm

Big Data. When I tell people that I’m hoping to study ‘Big Data’ and it’s impact on Agriculture as a part of my Nuffield Scholar travels for 2015, I’m greeted with one of 3 general responses:

  1. Most rare. If you’ve given me this response. Kudos to you. You, fellow nerd and I are friends.

‘Oh cool! Where are you hoping to travel?

  1. A little more common. But mostly because of the farming circles I’ve been hanging in

‘Oh good, I’ve got x number of years worth of farm data on the office computer and I don’t know what to do with it. It’s probably useful but I don’t have the skills to investigate it, and even if I did it’d be too time consuming.’ Yep I agree. It is time consuming and it is harder than it needs to be

  1. Most common response:

*Blank Stare* *Awkward pause* ‘What’s that?’

Well I’m so glad you asked.

Those who fit into category 1 probably think of something like this when they hear the words ‘data farm’:

One of Facebook’s data centers.

This is what I think of when I hear the words ‘data farm:’

View from the Silo - Sept 10 #2

Over the past decade it has become possible to easily and cheaply collect data about various aspects of our farms operation. Some of the many examples of this include the following:

  • Yield data – Tractors and harvesters have been sold with GPS guidance and steering as standard for around a decade now. This GPS enables yield information to be gathered and spatially mapped during harvest. We are even starting to see mapping of grain quality parameters such as moisture, oil content or protein levels.
  • Telemetry/Machine data – GPS data combined with recent advancements in telemetry has meant that farmers can choose to log machine performance data in real-time as it is happening in field. This means engine loading and fuel consumption data.
  • Em38 mapping. Em38 maps are a measure of electrical conductivity of soil. This can be mapped and gives us clues as to soil type variation across a paddock.
  • Weather data – The BOM makes it’s weather data from all its weather stations freely available to the public going back as far as records go. For my area this is back to 1883. Farmers in more remote locations can setup their own weather stations with data logging for only a few hundred dollars.
  • NDVI and aerial photography and mapping. A 2014 Nuffield Scholar is researching drones and their ability to be used on farm to capture these types of images. This can also be done from planes and satellites.
  • Paddock operation data – For our own record keeping we can collect data relating to what inputs we apply to the fields as they’re applied. This includes fertiliser and chemical records.
  • Soil moisture & temperature levels at various depths. To see how much water is available for plants.

These examples are biased towards my own industry of grain production, but I’ve spoken to a dairy farmer who has just started collecting individual production data for each cow he milks! He can record things like body temperature, milk production and daily movements. I’ve also met a horticulturalist who monitors radiation, humidity and temperature levels in his greenhouses.

This ability to collect and store all this farm related data presents many opportunities for farmers and also more than a few challenges.


  • Being better able to identify consistently over or under performing areas of the farm.
  • Having more accurate information when making decisions about machinery or infrastructure upgrades.
  • The possibility of creating value for farmers who generate data that could clearly be of interest to marketing and research companies.
  • Becoming more specific about the application of inputs such as seed, fertiliser and chemical, as opposed to making a broad assessment about what a large area of land needs.
  • Discovering things about our farm that we weren’t previously aware of.


  • Who owns or can access farmer generated data? Some current machinery manufacturers offer cloud based telemetry systems where farmer data is stored on the manufacturers servers.
  • Managing and storing these large amounts of data. Some of my friends struggle to place a phone call on their farms, let alone deal with mobile internet bandwidth restrictions.
  • Ease of use. Farm software is traditionally written by nerds. Most of it is hard for me to use and I have a tech background. Normal farmers without this experience often don’t have a hope of getting this stuff to work for them.
  • Justifying initial investment in time/money/effort to learn new skills to deal with this data. Donald Rumsfeld famously tried to explain, we do not know what we do not know. So it can be hard to sell or quantify the benefits of an unknown improvement.

It is all these issues and more that I’m hoping to explore during my studies next year. I’m really excited about this opportunity as I firmly believe this technology is coming to Agriculture in a big way. Last year the large agribusiness company Monsanto dumped nearly US $1 billion dollars on a company using climate data to help farmers manage weather risk. That sort of investment requires a fair bit of belief in a technology. We farmers can use this technology to profit ourselves, or we can wait until those we deal with use it to profit at our expense.

I was on a tech support call for some of our farm software the other day and the tech who was helping me made the comment ‘some of our new programmers who have come across from other industries (inc. mining and aviation) have been blown away by just how bloody complicated farming is’. He’s right. Farming is complicated. Which means huge amounts of data can be collected about it. As computing power gets cheaper and sensors become more available and easy to use big data will enable farmers to improve what they do, in ways they don’t yet foresee.


Web developer turned farmer. Interests include: my faith, my wife, technology, cricket, farming, ice cream & world events.

You may also like...

8 Responses

  1. Ainsley McArthur says:

    Have just discovered your blog! Congratulations on winning a Nuffield Scholarship. I guess I fit into the #2 field (have a lot of data on my computer type farmer) but am completely frustrated by this because I just know meaningful use of this data is the way forward in our industry (beef). I’m looking forward to following your journey … best of luck!

    • Jonathan says:

      Hi Ainsley,

      Thanks for stopping by! It just seems to me that agribusiness is moving into this area in a big way, and as the old saying goes ‘if you’re not sitting at the table you could be what’s on the menu’!

  2. Did you know Grain Growers Limited is the largest holder of Agricultural ‘Big Data’ in Australia. To put some perspective, GGL does everything that Climate Corp does in the US plus more. AFI are also proposing a research project to do the same thing Feel free to come and catch up if your ever in Canberra.

    • Jonathan says:


      Thanks so much for reading and commenting. Have added both you and the AFI to my ‘must visit’ list. I knew GGL was dabbling around in on-farm data through ProductionWise but had no idea about it’s interest in Big Data, the weather and insurance?

      Can’t wait for it all to happen!


  3. Darren says:

    Contemplating a trial to pull all these technologies together. Would like to discuss with you offline.

  4. Mark Pawsey says:

    SST Software is leaping into the Big Data space driven from ground up with our agX data standards platform we are now licensing to the industry. Big Data will be a big mess without A) standardized data and B) using data collected for business purpose and on farm value. Happy to discuss . Mark Pawsey

  5. Peter Warren says:

    Farm Records has a different approach to other software suppliers. It’s focus is on enabling researchers to get their science out there to show the value of the data. Farm Records uses a common platform to make it easier and more cost effective. This approach eliminates the duplication of effort, and allows the focus to be the innovation rather than the delivery vehicle. Data will flow but there has to be a reason and demonstrable benefits.

Leave a Reply

Your email address will not be published.