Median Filtering ADC data.




I recently observed ADC values from my microcontroller giving me frequent anomalous results magnitudes higher than expected.

For example when nothing was being measured and values should be near zero, however, the occasional 300+ value would show up. This could be attributed to noise in the system or PCB design issues etc.

I noticed the spike would only come in a single reading, meaning no two successive  reads would have spiked values. This may only have been the case because I was reading multiple channels and thus, by the time the ADC looped around to the problem channel the spike was gone, however if I had only been reading this channel continuously I surely would have seen the spike more than once in a row. However, since I indeed only saw a bad reading one time in succession, coupled with the fact that  the spikes were so much larger than readings before or after, the problem  lent  itself  to be solved with a simple median filter


Like all programming problems you can go down the rabbit-hole with this one as well. However after a couple of minutes of research I realize most advanced median filtering techniques are in the context of image filtering, or extremely large data sets. None of which apply to my scenario.


Having an extremely large dataset in my scenario would wreak havoc on the operation and safety of the device I was working with, taking 1000 readings and finding the median is not an option. This is why I opted for the simplest and probably fastest (to my knowledge) approach to the median filter. This is what we will explore. Get in losers we're going coding. 


Median Filter


The idea behind a media filter is as follows:

Given a dataset, sort it from least to greatest and find the median. Median being the value in the middle, not to be confused with mean which is the average.


Given the sequence of ADC readings seen in the figure below it is easy to spot the anomalous reading because it is highlighted in blue. However your readings will not always show up as blue. This is what the data on the device I was working on looked like, with even higher spikes as mentioned before, but for demonstration purposes 100 will be a good enough spike to show how far different it is from the surrounding readings. 


ADC FIFO





The next step is to sort your dataset from least to greatest. Be mindful that you do not want to sort your original FIFO. It is important, at least for my application, that the data is a true FIFO meaning that old data gets pushed out as new data comes in, so when I do my sorting I make a copy and sort the copy. However since the number of readings we are using is small it is not such a huge deal if you lose the integrity of your FIFO. It is clear to see that the median of the sorted data below is 6 and that the spike value has been pushed to one side.


ADC  data sorted




In the image below we add a new value to the original FIFO. Keeping track of a FIFO is a simple operation of % and your index size. The code below does a better job of explaining things. 



ADC FIFO, plus one new element and pushing out the oldest.

 




Now again I will make a copy and sort it to find the median, as seen below the median now is 5.


ADC data sorted.



If we go through this process for 3 more readings you will realize that the 100 value will get pushed out as new data comes in and the FIFO gets filled.


Through the process of the median filter our application will never use the spiked value because we will only be using the median which does a good job at filtering out the 100 value. 


And that's it folks. The code implements this by grabbing an ADC reading and passing it to a median filter function that returns the median. The code is below and I have commented well enough that I do not think it needs to be explained. 


I also will provide a better version of the code that you can save and use as a library and allows for more than one data set to be filtered while keeping track of each data set parameters with structs. I love structs.


Sorry I do not know why the entire code does not post, I tried several times. Here is a link to the github repo: Simple Median Filter Repo



This is the simple version that will simply filter the data and return the median.
#include <stdint.h>
#include <stdio.h>
#define FIFO_SIZE 5
#define MIDDLE_INDEX 3
#define swap(x,y) ((x ^= y),(y ^= x),(x ^= y))

uint32_t median_filter( uint32_t latest_value)
{
    
    static uint32_t local_ADC_FIFO_copy[FIFO_SIZE] = {0};
    static uint32_t index_counter = 0 ; 
    /*-------------| median filter|------------------------
        1.add lastest vaue & get rid of oldest value (fifo)
        2.make local copy
        3.Sort local copy only  ("bubble sort")
        4.Increment index counter for fifo
        5.return median
    */
    //1
    local_ADC_FIFO_copy[index_counter%FIFO_SIZE] = latest_value;

    //2
    uint32_t temp_sorted[FIFO_SIZE] = {0}; //make a temp to store sorted version
    for(int i = 0 ; i < FIFO_SIZE  - 1; i++)
    {
        temp_sorted[i] = local_ADC_FIFO_copy[i];
    }
    //3
    uint8_t counter = FIFO_SIZE;
    while(counter)
    {
        for(int i = 0 ; i < (counter - 1) ; i++)
        {

            if(temp_sorted[i] > temp_sorted[i+1])
            {   //swap
                swap(temp_sorted[i] ,temp_sorted[i+1]) ;
            }
           
        }
        counter--;
    }
    //---------------
    //4
    index_counter++;
    //5
    return (uint32_t)temp_sorted[MIDDLE_INDEX]; 

}


int main(void)
{
    //each element here can be a new adc reading coming in
    uint32_t adc_data[20] = {1,2,3,425,2,1,3,4,2,1,1,2,3,4,3,2,1,4,5,6} ;

    for(int i = 0 ; i < 20; i++)
    {
        printf("Median : %d \n",median_filter(adc_data[i]));
    }

}

 This version gives you the ability to filter more than one data set each with its own parameters.  
#include <stdint.h>
#include <stdio.h>


#define swap(x,y) ((x ^= y),(y ^= x),(x ^= y))

typedef struct
{
   uint32_t          *adc_fifo;            //this is where the local copy will be stored 
   uint8_t            fifo_width;
   uint8_t            median_index;
   uint8_t            internal_counter; 
} ADCFilterdValues_T;



uint32_t median_filter(ADCFilterdValues_T *adc_filter , uint32_t latest_value)
{
    /*-------------| median filter|------------------------
        1.add lastest vaue & get rid of oldest value (fifo)
        2.make local copy
        3.Sort local copy only  ("bubble sort")
        4.Increment index counter for fifo
        5.return median
    */
    //1
    adc_filter->adc_fifo[adc_filter->internal_counter%adc_filter->fifo_width] = latest_value;
    
    //2
    uint32_t temp_sorted_raw[adc_filter->fifo_width] ; 
    for(int i = 0 ; i < adc_filter->fifo_width ; i++)
    {
        temp_sorted_raw[i] = adc_filter->adc_fifo[i];
    }

    //3
    uint8_t counter = adc_filter->fifo_width;
    while(counter)
    {
        for(int i = 0 ; i < (counter - 1) ; i++)
        {

            if(temp_sorted_raw[i] > temp_sorted_raw[i+1])
            {   //swap
                swap(temp_sorted_raw[i] ,temp_sorted_raw[i+1]) ;
            }
        }
        counter--;
    }
    //4
    adc_filter->internal_counter++;
    //5
    return (uint32_t)temp_sorted_raw[adc_filter->median_index]; 

}

void main(void)
{
    //simualted stream of data coming in
    uint32_t adc_data_1[20] = {1,2,3,425,2,1,3,4,2,1,1,2,3,4,3,2,1,4,5,6} ;
    uint32_t adc_data_2[40] = {1,2,3,425,2,1,3,4,2,1,1,2,300,4,3,2,1,4,5,6,1,2,3,425,2,1,3,4,2,1,100,2,3,4,3,2,1,4,5,6} ;

    //make instances of median filter type
    ADCFilterdValues_T myFilter_1 ;
    uint32_t adc_fifo_1[5] = {0}; 

    ADCFilterdValues_T myFilter_2 ;
    uint32_t adc_fifo_2[11] = {0};

    //configure the filter
    myFilter_1.adc_fifo = adc_fifo_1;
    myFilter_1.fifo_width = 5;
    myFilter_1.median_index = 2; 

    myFilter_2.adc_fifo = adc_fifo_2;
    myFilter_2.fifo_width = 11;
    myFilter_2.median_index = 5; 

    for(int i = 0 ; i < 20; i++)
    {
        //call median filter and give it instance 1 and feed it fake data from adc_data_1
        printf("Median : %d \n",median_filter(&myFilter_1 ,adc_data_1[i] ));
    }
    printf("-----\n");
    for(int i = 0 ; i < 40; i++)
    {
        //call median filter and give it instance 1 and feed it fake data from adc_data_1
        printf("Median : %d \n",median_filter(&myFilter_2 ,adc_data_2[i] ));
    }

}



Comments

Share your comments with me

Archive

Contact Form

Send