Skip to content

Instantly share code, notes, and snippets.

@skjerns
Last active October 20, 2024 03:09
Show Gist options
  • Save skjerns/bc660ef59dca0dbd53f00ed38c42f6be to your computer and use it in GitHub Desktop.
Save skjerns/bc660ef59dca0dbd53f00ed38c42f6be to your computer and use it in GitHub Desktop.
Save a mne.io.Raw object to EDF/EDF+/BDF/BDF+
# -*- coding: utf-8 -*-
"""
Created on Wed Dec 5 12:56:31 2018
@author: skjerns
Gist to save a mne.io.Raw object to an EDF file using pyEDFlib
(https://github.com/holgern/pyedflib)
Disclaimer:
- Saving your data this way will result in slight
loss of precision (magnitude +-1e-09).
- It is assumed that the data is presented in Volt (V),
it will be internally converted to microvolt
- BDF or EDF+ is selected based on the filename extension
- Annotations preserved
Update: Since 2021, MNE also supports exporting EDF via edfio:
https://mne.tools/stable/generated/mne.export.export_raw.html
"""
import pyedflib # pip install pyedflib
from pyedflib import highlevel # new high-level interface
from pyedflib import FILETYPE_BDF, FILETYPE_BDFPLUS, FILETYPE_EDF, FILETYPE_EDFPLUS
from datetime import datetime, timezone, timedelta
import mne
import os
def _stamp_to_dt(utc_stamp):
"""Convert timestamp to datetime object in Windows-friendly way."""
if 'datetime' in str(type(utc_stamp)): return utc_stamp
# The min on windows is 86400
stamp = [int(s) for s in utc_stamp]
if len(stamp) == 1: # In case there is no microseconds information
stamp.append(0)
return (datetime.fromtimestamp(0, tz=timezone.utc) +
timedelta(0, stamp[0], stamp[1])) # day, sec, μs
def write_mne_edf(mne_raw, fname, picks=None, tmin=0, tmax=None,
overwrite=False):
"""
Saves the raw content of an MNE.io.Raw and its subclasses to
a file using the EDF+/BDF filetype
pyEDFlib is used to save the raw contents of the RawArray to disk
Parameters
update 2021: edf export is now also supported in MNE:
https://mne.tools/stable/generated/mne.export.export_raw.html
----------
mne_raw : mne.io.Raw
An object with super class mne.io.Raw that contains the data
to save
fname : string
File name of the new dataset. This has to be a new filename
unless data have been preloaded. Filenames should end with .edf
picks : array-like of int | None
Indices of channels to include. If None all channels are kept.
tmin : float | None
Time in seconds of first sample to save. If None first sample
is used.
tmax : float | None
Time in seconds of last sample to save. If None last sample
is used.
overwrite : bool
If True, the destination file (if it exists) will be overwritten.
If False (default), an error will be raised if the file exists.
"""
print('did you know EDF export is now supported in MNE via edfio? have a look at https://mne.tools/stable/generated/mne.export.export_raw.html')
if not issubclass(type(mne_raw), mne.io.BaseRaw):
raise TypeError('Must be mne.io.Raw type')
if not overwrite and os.path.exists(fname):
raise OSError('File already exists. No overwrite.')
# static settings
has_annotations = True if len(mne_raw.annotations)>0 else False
if os.path.splitext(fname)[-1] == '.edf':
file_type = FILETYPE_EDFPLUS if has_annotations else FILETYPE_EDF
dmin, dmax = -32768, 32767
else:
file_type = FILETYPE_BDFPLUS if has_annotations else FILETYPE_BDF
dmin, dmax = -8388608, 8388607
print('saving to {}, filetype {}'.format(fname, file_type))
sfreq = mne_raw.info['sfreq']
date = _stamp_to_dt(mne_raw.info['meas_date'])
if tmin:
date += timedelta(seconds=tmin)
# no conversion necessary, as pyedflib can handle datetime.
#date = date.strftime('%d %b %Y %H:%M:%S')
first_sample = int(sfreq*tmin)
last_sample = int(sfreq*tmax) if tmax is not None else None
# convert data
channels = mne_raw.get_data(picks,
start = first_sample,
stop = last_sample)
# convert to microvolts to scale up precision
channels *= 1e6
# set conversion parameters
n_channels = len(channels)
# create channel from this
try:
f = pyedflib.EdfWriter(fname,
n_channels=n_channels,
file_type=file_type)
channel_info = []
ch_idx = range(n_channels) if picks is None else picks
keys = list(mne_raw._orig_units.keys())
for i in ch_idx:
try:
ch_dict = {'label': mne_raw.ch_names[i],
'dimension': mne_raw._orig_units[keys[i]],
'sample_rate': mne_raw._raw_extras[0]['n_samps'][i],
'physical_min': mne_raw._raw_extras[0]['physical_min'][i],
'physical_max': mne_raw._raw_extras[0]['physical_max'][i],
'digital_min': mne_raw._raw_extras[0]['digital_min'][i],
'digital_max': mne_raw._raw_extras[0]['digital_max'][i],
'transducer': '',
'prefilter': ''}
except:
ch_dict = {'label': mne_raw.ch_names[i],
'dimension': mne_raw._orig_units[keys[i]],
'sample_rate': sfreq,
'physical_min': channels.min(),
'physical_max': channels.max(),
'digital_min': dmin,
'digital_max': dmax,
'transducer': '',
'prefilter': ''}
channel_info.append(ch_dict)
f.setPatientCode(mne_raw._raw_extras[0]['subject_info'].get('id', '0'))
f.setPatientName(mne_raw._raw_extras[0]['subject_info'].get('name', 'noname'))
f.setTechnician('mne-gist-save-edf-skjerns')
f.setSignalHeaders(channel_info)
f.setStartdatetime(date)
f.writeSamples(channels)
for annotation in mne_raw.annotations:
onset = annotation['onset']
duration = annotation['duration']
description = annotation['description']
f.writeAnnotation(onset, duration, description)
except Exception as e:
raise e
finally:
f.close()
return True
@MAXMT75
Copy link

MAXMT75 commented May 12, 2020

Good afternoon there.

I was wondering whether you managed to keep the annotations in saving raw data to a edf.file

Thanks.
M.

@skjerns
Copy link
Author

skjerns commented May 12, 2020

@MM75CO yes it does that.

@LunaHub
Copy link

LunaHub commented Jun 29, 2020

Hi,

Great work!! I really miss this option in the mne package :D

I am trying to use the gist to convert a csv file to edf. I have contructed my own mne raw object from the data in the csv file, but I get an error when trying to save the Raw object to edf. I get the following error and the output file is empty (size 0KB):
'NoneType' object has no attribute 'keys'
The error seems to originate from line 103 and mne_raw._orig_units?

I also get this error when converting the timestamp. But I think solved this by omitting it. (But maybe that is messing something else up?)
'NoneType' object is not iterable

@alexrockhill
Copy link

@LanaHub You might want to see what the mne field is for the example mne data and just copy from there.

@skjerns do you think there is any interesting in making this a PR to mne? There is a lot of i/o structure in mne already but it's all based around loading data, I think it would be a bit of a new direction to save data in non-fif formats. MNE-BIDS already uses pybv because fif is not one of the BIDS required formats for EEG.

@skjerns
Copy link
Author

skjerns commented Jun 30, 2020

@LanaHub if you just want to convert a csv to edf (and do not need mne functionality), just use pyedflib directly! the highlevel interface should make that quite easy. Load the CSV with numpy or pandas and save it with pyedflib.

https://github.com/holgern/pyedflib#highlevel-interface

@alexrockhill I'd gladly do it, but the devs of mne do not want any third-party dependencies (which using pyedflib would introduce), that's why they told me to create a gist instead. That would mean that I'd have to create the edf creation from scratch in raw python. See also mne-tools/mne-python#5755

@LunaHub
Copy link

LunaHub commented Jun 30, 2020

Thank you for getting back so soon!
I would like to use the mne functionallity, but I also want to save the data as edf to be compliant with BIDS. Btw. I get the same error when trying to convert a gdf file to edf..
I will have another look at the example mne data and try to figure out a way to make it work.

@alexrockhill
Copy link

alexrockhill commented Jun 30, 2020

@skjerns ahh, I see. Maybe worth thinking about whether this could be in mne-bids since there is already a pybv dependency. That does seem like when the data needs to be in edf, i.e. when it's in BIDS. @jasmainak what do you think?

Also, the gist is nice, thanks for doing that but it is a rather important part that can go very wrong if the file is exported incorrectly and it would be great to have some round trip tests that would be added if it were to be a PR.

@jasmainak
Copy link

I'm a bit hesitant to take on the responsibility of an edf writer. There are tons of corner cases in the edf format. Why not contribute to existing efforts such as pyedf: https://github.com/bids-standard/pyedf

@alexrockhill
Copy link

That doesn't have any tests either haha

It seems like there is a lot of use of the gist and like people would probably like it to be officially supported by mne and @skjerns said he would write the initial PR but I understand the hesitance and I get the message. I thought it was worth floating the idea.

@datalw
Copy link

datalw commented Aug 21, 2020

Hi,
this gist is really great idea! Just the functionality, which I have been searched in mne and did not find. A great supplement. Big thanks to you!

I tried to save the preprocessed mne raw object into a new edf, and got the following error
...

line 27, in _stamp_to_dt
    stamp = [int(s) for s in utc_stamp]
TypeError: 'datetime.datetime' object is not iterable

raw.info['meas_date'] in my script gives me:

datetime.datetime(2020, 3, 12, 16, 34, 55, tzinfo=datetime.timezone.utc)

Have you encoutered this problem?

@LunaHub
Copy link

LunaHub commented Aug 21, 2020

Yes, I had the same problem. I solved it by changing line 76 to date = mne_raw.info['meas_date']. However then I got a new problem and I actually gave up :P Let me know if you succed!

@skjerns
Copy link
Author

skjerns commented Aug 21, 2020

I've adapted the script.

@datalw can you check if it works now?

@datalw
Copy link

datalw commented Aug 21, 2020

@skjerns Thanks a lot for the swift response! I tried the adapted version and got the following error:

----> 3     if 'datetime' in type(utc_stamp): return utc_stamp
      4     # The min on windows is 86400
      5     stamp = [int(s) for s in utc_stamp]

TypeError: argument of type 'type' is not iterable

So I changed if 'datetime' in type(utc_stamp): to if 'datetime' in str(type(utc_stamp)) and it works :D Would you mind to adapt it again?
Thanks a lot for the great work!
@LunaHub it works for me so far, you could try it again : )

@skjerns
Copy link
Author

skjerns commented Aug 21, 2020

oops, yes didnt test it, thank's, I corrected it

@datalw
Copy link

datalw commented Sep 14, 2020

I have used the gist for a while, it is really awesome, especially for saving the preprocessed data!

I am here again for a follow-up question. So far the files without annotations have been saved successfully. However for a file with annotations, the raw.times in the saved file was incorrect. I have looked into the code and found out the problem might be due to this line
'sample_rate': mne_raw._raw_extras[0]['n_samps'][i]

mne_raw.info['sfreq'] gives 256, but mne_raw._raw_extras[0]['n_samps'][i] gives 2048 instead.
Since I have never dealt with _raw_extras, I tried to find some information about _raw_extras in mne documention, but failed.

Does anyone know how to solve this problem, or how I can proceed? Or is this a bug in _raw_extras?

@alexrockhill
Copy link

alexrockhill commented Sep 14, 2020

I am not familiar with raw._raw_extras you might be able to reach a wider audience that wrote that part of the code on Gitter unless someone on this thread knows https://gitter.im/mne-tools/mne-python

@skjerns
Copy link
Author

skjerns commented Sep 16, 2020

@datalw did you resample the data? was one of original sample frequencies 2048?

to be honest, I just hacked this snippet together and changed it often over time, I'm also not very familiar with the mne internals. If you find a solution, let me know! I'm still in favor of integrating edf writing compatibility inside mne, but currently they don't want to introduce optional dependencies. maybe that changes as a python-native implementation of edflib has been published last month :)

@nateGeorge
Copy link

So this still isn't incorporated into mne-python, any chance it ever will? Been a few years.

@skjerns
Copy link
Author

skjerns commented Sep 28, 2020

Feel free to ask this directly at MNE: https://github.com/mne-tools/mne-python/issues or the gitter https://gitter.im/mne-tools/mne-python

The problem was the additional dependency on pyedflib and that pyedflib uses Cython to compile some C libraries, which they did not want to include (afaik, and which I somehow understand).

However, we could ask if it would be possible to include an optional dependency on pyedflib. Additionally, EDDlib has just released a Python-only-version which would solve some problems :) but its quite slow (it's Python, nevertheless)

@datalw
Copy link

datalw commented Sep 28, 2020

@alexrockhill @skjerns Thanks a lot for the reply and sorry for my delayed answer. I plan to come back to this issue in the next couple of weeks and if I find a solution I will let you @skjerns know ; )

@bonniekcn
Copy link

Hello, thanks a lot for the great work! I am having one problem when using this script. When I try to open the resulted edf file this message came up "Error, number of datarecords is 0, expected >0. You can fix this problem with the header editor, check the manual for details. File is not a valid EDF or BDF file.". I am not quite sure why is this the case and I wonder if anyone is having the same problem or has any solution.

Thanks a lot!

@alexrockhill
Copy link

That's a bit hard to diagnose what's going on without sharing a minimally reproducible example that someone else can run on their machine.

It sounds like maybe your mne.io.Raw object didn't have any data in it or any data of type eeg, grad, mag or seeg. See https://mne.tools/stable/generated/mne.io.Raw.html#mne.io.Raw.set_channel_types.

@bonniekcn
Copy link

That's a bit hard to diagnose what's going on without sharing a minimally reproducible example that someone else can run on their machine.

It sounds like maybe your mne.io.Raw object didn't have any data in it or any data of type eeg, grad, mag or seeg. See https://mne.tools/stable/generated/mne.io.Raw.html#mne.io.Raw.set_channel_types.

@alexrockhill Hello, thanks a lot for the reply. This is the piece of code I used when creating the mne.io.Raw object.
sample_data_raw_file = os.path.join("filename.edf")
raw = mne.io.read_raw_edf(sample_data_raw_file)
My original document is already in .edf, I put the file through some processing in mne-python and I would want to save the end-product as .edf using your script. Thanks a lot for your help!

@alexrockhill
Copy link

alexrockhill commented Oct 16, 2020

From my experience reading in edf files, often the data types are not correctly set by mne by default. I would try setting all your channel types to eeg I assume but whatever type they are and saving again. I'd be interested to know if that works.

Also, I helped a bit but the thanks definitely goes to @skjerns for this gist.

Maybe this will help

import mne
raw_fname = 'filename.edf'
raw = mne.io.read_raw_edf(raw_fname)
raw = raw.set_channel_types({ch: 'eeg' for ch in raw.ch_names})

@bonniekcn
Copy link

@alexrockhill
Thanks for the reply. I have tried setting the channel types to eeg using raw = raw.set_channel_types({ch: 'eeg' for ch in raw.ch_names}) , but the resulting file still can't be opened.

@brainhong913
Copy link

I am having the same. My guess is that there is some unexpected bug occurs while you reading the raw data from edf. Is there any method to sure I read the edf correctly?

@alexrockhill
Copy link

Ok, that was just an informed guess but I couldn't tell you what's wrong because I don't have access to your data.

@bonniekcn
Copy link

I realized that the smaller the original file I use the smaller the resulting file that is saved. But nomatter how big my original file is, the resulting file is always too small to be opened. When I called the function I called it this way write_mne_edf(raw,'testfile.edf', overwrite=True). I wonder if I am not aware of some details from the script resulting in this error of getting an extremely small file.

@skjerns
Copy link
Author

skjerns commented Oct 20, 2020

It is difficult to help you without having the original file to work with. Feel free to upload the file somewhere (eg wetransfer) and post the link here. Alternatively, if the data is sensitive, send it privately to [email protected]

@bonniekcn
Copy link

It is difficult to help you without having the original file to work with. Feel free to upload the file somewhere (eg wetransfer) and post the link here. Alternatively, if the data is sensitive, send it privately to [email protected]

Hello, thank you very much for your reply. I have sent an email to the address provided. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment