Skip to content

Dairy Farming Is Getting a Big Data Boost

    Cows in green field, depiction of dairy farming

    Written by: Marina Wang | Issue # 105 | 2021

    • Data sets from dairy farms are being cleaned, standardized, and integrated to create a suite of new tools to assist dairy farmers in decision making.
    • Tools include calculating feed efficiency, determining nutritional groupings, detecting early onset for clinical mastitis, determining the financial value of a cow, and creating first lactation curves.
    • Farmers will be able to easily access these customizable tools in a web platform or app.

    Industries around the world are being swept up in a Big Data and AI revolution, and dairy is no exception. A new, multidisciplinary project called Dairy Brain is using big data analytics and AI to give the industry a technological boost. Dairy Brain, a collaborative project with the University of Wisconsin-Madison, is designing a web-based platform with a suite of smart tools informed by data analytics and AI that assists dairy farmers in management and decision-making.

    “The main idea of the Dairy Brain project is to collect data from farms, integrate all the data, and develop decision-supporting tools at different levels,” says Liliana Fadul-Pacheco, a research associate at the University of Wisconsin-Madison.

    To start, Fadul-Pacheco and a multi-disciplinary team of collaborators collected data from a variety of sources, from the milking parlor to the Dairy Herd Improvement Assocation (DHIA), to genetics. They then set to work cleaning, standardizing, and integrating the information in a portion of the project called the Agricultural Data Hub. With these gargantuan quantities of data they then have been able to create a host of tools to assist dairy farm management [1].

    “This type of data has been always there, but I think the nice thing about what we’re doing at Dairy Brain—the innovation part—is integrating everything,” says Fadul-Pacheco, explaining that while farmers have lots of raw data and statistics, they don’t necessarily have the time or expertise to analyze it. One of the early challenges, she says, is that streams of information historically have been kept separate and being able to link and standardize the data will lead to a more robust dataset that can train machine learning algorithms.

    For example, Fadul-Pacheco says farmers may have feed data and milk production data, but the two might not be integrated in order to calculate feed efficiency, a ratio between the amount of feed a cow is consuming and her milk production. By integrating the two datasets, farmers can get quick and up-to-date feed efficiency statistics for their herds. Anomalies in feed efficiency could indicate to the farmer a possible issue with the feed or feeding regimen, for example.

    And the feed efficiency calculator is just one simple tool created using dairy’s Big Data. Another tool has been created for nutritional group management. When feeding cows, farmers can group them by parameters such as lactation stage, age, or size, but a new algorithm can take all these factors into account and create the optimal groupings for the farmer [2].

    Fadul-Pacheco is also building a machine-learning algorithm that can predict which cows might be at risk of developing clinical mastitis, an infectious disease of the mammary glands that causes visibly abnormal milk. The machine-learning algorithm can be trained on large datasets to scan for patterns and predictive variables that could be early indicators of the disease. The many variables and statistical biases in the data are a challenge to work around, and Fadul-Pacheco says, “The next steps right now are to have more data, continue training the algorithm, and maybe add some genetic variables” [3].

    Another complex tool is designed to calculate the financial value of a cow. “If you have two cows and you need to sell one because you don’t have space, you need to decide which will be better in the future,” Fadul-Pacheco offers as a simple example. By considering variables such as productivity, health, and reproduction, Dairy Brain researchers can create an algorithm for estimating the net financial value of each cow in order to inform management decisions.

    Another module, which Fadul-Pacheco says is still in the early stages of development, is calculating the first lactation curve for cows. Lactation curves are created when a cow gives birth and her milk productivity is graphed over time. Dairy farmers can create the curves for cows that have had at least one calf, as the data is extrapolated from their first birth. However, determining a first lactation curve for a cow is trickier. Data scientists are working on an algorithm to calculate this based on a cow’s siblings.

    “It’s been challenge after challenge, but we’re getting there,” says Fadul-Pacheco, who expects Dairy Brain to be ready for use in the next year or two. Other aspects of Dairy Brain have included the Coordinated Innovation Network (CIN) that guides the responsible use of data, and the Extension Program, which regularly collaborates with dairy farmer stakeholders on the design and practicality of tools. “There are tools that are coming that are super practical,” says Fadul-Pacheco.


    1. Cabrera, V. E and L. Fadul-Pacheco. 2021. Future of dairy farming from the Dairy Brain perspective: Data integration, analytics, and applications. International Dairy Journal 121:105069.

    2. Barrientos, J. A., H. White, R. D. Shaver and V. E. Cabrera. 2020. Improving nutritional accuracy and economics through multiple ration-grouping strategy. Journal of Dairy Science 103:3774-3785.

    3. Fadul-Pacheco, L., H. Delgado and V. E. Cabrera. 2021. Exploring machine learning algorithms for early prediction of clinical mastitis. International Dairy Journal 119:105051.