Getting timestamp from the pandas dataframe

[markusmat97]

I converted the mdf file into csv. Now I want to plot a channel sample versus time using the pandas dataframe.How to do this?

I have plotted the sample vs time using get_can_signal.Now I want to try it with pandas datgaframe.On converting to csv,I didnt,see the timestamp in the dataframe.
I would like to know:

• How to get the timestamp in pandas dataframe/csv file?
  

• How to plot a channel sample versus time from pandas dataframe?

Thank you very much in advance

[danielhrisca]

I'm really not great at using pandas so I don't know what facilities it provides for plotting. The timestamps are found in the dataframe index.

[markusmat97]

How to access that index?Why it is not visible in dataframe when mdf is converted to csv?

[danielhrisca]

So you first convert to CSV, and after that you load the CSV into a pandas dataframe? Why not use MDF.to_daframe directly?

[markusmat97]

i used the function MDF.to_dataframe for converting to dataframe.but wheni saved it as csv i didn't find the timestamp in CSV file and python interpolates many row values.

[david32]

Hi,

So, when converting to dataframe, the timestamps become the index of the Data Frame. you can still use .plot() and specify your y axis with the name of your column.
Notice that if you use the default "use_interpolation = True" it will interpolate in the "missing" values. if you set it to False, they will be set to NaN, and then you can't plot the colums that contain those values. (At least it doesn't work for me).

But otherwise, you don't have to convert to csv to plot your signal, convert the MDF to Dataframe, and plot using the signal name.
My example is kind of silly but works:

mdf = MDF()
mdf.append(Signal(name="test", samples= np.random.rand(20), timestamps= np.arange(0,10,0.5)))
mdf.append(Signal(name="ECM_test2", samples= np.random.rand(40) + 5, timestamps= np.arange(0,10,0.25)))

mdf_df = mdf.to_dataframe()
mdf_df.plot(y='test')
plt.show()
mdf_df.plot(y='ECM_test2')
plt.show()

Perhaps if you would describe what you need to do, we can find a better way to do it.
If I add:

mdf_df.to_csv(f'C:\\test.csv')

I get 3 columns as expected: timestamps, test, ECM_test2
And then when reading from it, I can access the timestamps column directly.

[also, converting and saving to csv is highly inefficient in my opinion. you should do it only if you have to or with a very small file]

[markusmat97]

@david32

1. I was working with an Mdf file(sample:https://docs.google.com/spreadsheets/d/1x9dGkjGLfzGb-yN1RTNEf4LsQJ8I1Rox-pzRfR8FOWQ/edit#gid=0).It has many channel for example Azi_0,Azi_1,Azi_2.... so on.I want to filter the data first based on Azimuth to avoid irrelevant detection(condition1:-1.047198<= Azi >=1.047198) then next with distance(0 to 20)
2. I want to know is there any way in asammdf for filtering the data,said in 1.I know i can filter the mdf file based on mdf.filter(channel:channel 1)
3. currently i am converting the mdf file to dataframe(mdf.to_dataframe) for filtering and plotting.Do you know any efficient way other converting pandas dataframe?

[david32]

The sample is already filtered from what I see, correct?

I am no expert, and perhaps @danielhrisca can propose a different workflow.
What I would do is this:
Depending on your needs I would sync the similar channels (Azimuth / Distance).
Another option is the sync the whole mdf by a common raster.
You can do that by:

Azi0 = mdf.get("RUT1_Azi_meas_0")
Azi1 = mdf.get("RUT1_Azi_meas_1")

Azi1 = Azi1.interp(Azi0.timestamps)

You can filter the samples however you like, as they are stored in a numpy array:

Azi0.samples[Azi0.samples >= 1.047198] = 0 
Azi0.samples[Azi0.samples <= -1.047198] = 0 

You can act with these arrays however you like. The result can be plotted using matplotlib (which is actually the same as through pandas)