Skip to content

Instantly share code, notes, and snippets.

@fomightez

fomightez/testing_methylprep_updated_to_work_with_Pandas2.ipynb

Created July 5, 2024 15:57
Show Gist options
  • Save fomightez/adfcb07a9f00430057557473a5ef69d7 to your computer and use it in GitHub Desktop.
Save fomightez/adfcb07a9f00430057557473a5ef69d7 to your computer and use it in GitHub Desktop.
Download ZIP
Testing methylprep that someone adapted for Pandas 2 works. 7-6-2024 For SO https://stackoverflow.com/q/78668908/8508004
Display the source blob
Display the rendered blob
Raw
testing_methylprep_updated_to_work_with_Pandas2.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Testing methylprep that someone adapted for Pandas 2 works\n",
"\n",
"Associated with SO https://stackoverflow.com/q/78668908/8508004\n",
"\n",
"Ran this in a Jupyter session obtained by going [here](https://github.com/fomightez/3Dscatter_plot_mod_playground-binder) and clicking 'Launch binder' badge. (Used there because I know it comes with more updated Pandas then main, general Binder example I use.)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Equivalent of this had been run off in terminal in session\n",
"%pip install git+https://github.com/gilgameshjw/methylprep/@pandas2.0_0"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Showing Pandas 2.2.2 is installed and seeing if works!"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Package Version\n",
"------------------------- --------------\n",
"alembic 1.13.1\n",
"anyio 4.3.0\n",
"argon2-cffi 23.1.0\n",
"argon2-cffi-bindings 21.2.0\n",
"arrow 1.3.0\n",
"asttokens 2.4.1\n",
"async-generator 1.10\n",
"async-lru 2.0.4\n",
"attrs 23.2.0\n",
"Babel 2.14.0\n",
"beautifulsoup4 4.12.3\n",
"bleach 6.1.0\n",
"blinker 1.7.0\n",
"Brotli 1.1.0\n",
"bs4 0.0.2\n",
"cached-property 1.5.2\n",
"certifi 2024.2.2\n",
"certipy 0.1.3\n",
"cffi 1.16.0\n",
"charset-normalizer 3.3.2\n",
"click 8.1.7\n",
"cloudpickle 3.0.0\n",
"colorcet 3.1.0\n",
"colorlover 0.3.0\n",
"comm 0.2.2\n",
"contourpy 1.2.1\n",
"cryptography 42.0.5\n",
"cufflinks 0.17.3\n",
"cycler 0.12.1\n",
"dask 2024.6.2\n",
"datashader 0.16.2\n",
"debugpy 1.8.1\n",
"decorator 5.1.1\n",
"defusedxml 0.7.1\n",
"entrypoints 0.4\n",
"exceptiongroup 1.2.0\n",
"executing 2.0.1\n",
"fastjsonschema 2.19.1\n",
"fonttools 4.53.0\n",
"fqdn 1.5.1\n",
"fsspec 2024.6.1\n",
"greenlet 3.0.3\n",
"h11 0.14.0\n",
"h2 4.1.0\n",
"hpack 4.0.0\n",
"httpcore 1.0.5\n",
"httpx 0.27.0\n",
"hyperframe 6.0.1\n",
"idna 3.6\n",
"imageio 2.34.2\n",
"importlib_metadata 7.1.0\n",
"importlib_resources 6.4.0\n",
"ipykernel 6.29.3\n",
"ipympl 0.9.4\n",
"ipython 8.22.2\n",
"ipython-genutils 0.2.0\n",
"ipywidgets 8.1.2\n",
"isoduration 20.11.0\n",
"jedi 0.19.1\n",
"Jinja2 3.1.3\n",
"json5 0.9.24\n",
"jsonpointer 2.4\n",
"jsonschema 4.21.1\n",
"jsonschema-specifications 2023.12.1\n",
"jupyter_client 7.4.9\n",
"jupyter_core 5.7.2\n",
"jupyter-events 0.10.0\n",
"jupyter-lsp 2.2.4\n",
"jupyter-offlinenotebook 0.2.2\n",
"jupyter-resource-usage 1.0.2\n",
"jupyter_server 2.13.0\n",
"jupyter_server_terminals 0.5.3\n",
"jupyter-telemetry 0.1.0\n",
"jupyterhub 3.1.1\n",
"jupyterlab 4.2.3\n",
"jupyterlab_pygments 0.3.0\n",
"jupyterlab_server 2.27.2\n",
"jupyterlab_widgets 3.0.10\n",
"kiwisolver 1.4.5\n",
"lazy_loader 0.4\n",
"llvmlite 0.43.0\n",
"locket 1.0.0\n",
"lxml 5.2.2\n",
"Mako 1.3.2\n",
"MarkupSafe 2.1.5\n",
"matplotlib 3.9.0\n",
"matplotlib-inline 0.1.6\n",
"methylprep 1.7.1\n",
"mistune 3.0.2\n",
"multipledispatch 1.0.0\n",
"nbclassic 1.0.0\n",
"nbclient 0.10.0\n",
"nbconvert 7.16.3\n",
"nbformat 5.10.3\n",
"nbgitpuller 1.2.1\n",
"nest_asyncio 1.6.0\n",
"networkx 3.3\n",
"notebook 7.2.1\n",
"notebook_shim 0.2.4\n",
"numba 0.60.0\n",
"numpy 2.0.0\n",
"oauthlib 3.2.2\n",
"overrides 7.7.0\n",
"packaging 24.0\n",
"pamela 1.1.0\n",
"pandas 2.2.2\n",
"pandocfilters 1.5.0\n",
"param 2.1.1\n",
"parso 0.8.3\n",
"partd 1.4.2\n",
"patsy 0.5.6\n",
"pexpect 4.9.0\n",
"pickleshare 0.7.5\n",
"pillow 10.4.0\n",
"pip 24.0\n",
"pkgutil_resolve_name 1.3.10\n",
"platformdirs 4.2.0\n",
"plotly 5.22.0\n",
"plotly-geo 1.0.0\n",
"prometheus_client 0.20.0\n",
"prompt-toolkit 3.0.42\n",
"psutil 5.9.8\n",
"ptyprocess 0.7.0\n",
"pure-eval 0.2.2\n",
"pyarrow 16.1.0\n",
"pycparser 2.21\n",
"pyct 0.5.0\n",
"Pygments 2.17.2\n",
"PyJWT 2.8.0\n",
"pyOpenSSL 24.0.0\n",
"pyparsing 3.1.2\n",
"PySocks 1.7.1\n",
"python-dateutil 2.9.0\n",
"python-json-logger 2.0.7\n",
"pytz 2024.1\n",
"PyYAML 6.0.1\n",
"pyzmq 24.0.1\n",
"referencing 0.34.0\n",
"requests 2.31.0\n",
"rfc3339-validator 0.1.4\n",
"rfc3986-validator 0.1.1\n",
"rpds-py 0.18.0\n",
"ruamel.yaml 0.18.6\n",
"ruamel.yaml.clib 0.2.8\n",
"scikit-image 0.24.0\n",
"scipy 1.14.0\n",
"seaborn 0.13.2\n",
"Send2Trash 1.8.2\n",
"setuptools 69.2.0\n",
"six 1.16.0\n",
"sniffio 1.3.1\n",
"soupsieve 2.5\n",
"SQLAlchemy 2.0.29\n",
"stack-data 0.6.2\n",
"statsmodels 0.14.2\n",
"tenacity 8.4.2\n",
"terminado 0.18.1\n",
"tifffile 2024.6.18\n",
"tinycss2 1.2.1\n",
"tomli 2.0.1\n",
"toolz 0.12.1\n",
"tornado 6.4\n",
"tqdm 4.66.4\n",
"traitlets 5.14.2\n",
"types-python-dateutil 2.9.0.20240316\n",
"typing_extensions 4.10.0\n",
"typing-utils 0.1.0\n",
"tzdata 2024.1\n",
"uri-template 1.3.0\n",
"urllib3 2.2.1\n",
"voila 0.5.7\n",
"wcwidth 0.2.13\n",
"webcolors 1.13\n",
"webencodings 0.5.1\n",
"websocket-client 1.7.0\n",
"websockets 12.0\n",
"wheel 0.43.0\n",
"widgetsnbextension 4.0.10\n",
"xarray 2024.6.0\n",
"xlrd 2.0.1\n",
"zipp 3.17.0\n",
"Note: you may need to restart the kernel to use updated packages.\n"
]
}
],
"source": [
"%pip list"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**The repo contains test data it seems!!** Trying it with that."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cloning into 'methylprep'...\n",
"remote: Enumerating objects: 1643, done.\u001b[K\n",
"remote: Counting objects: 100% (321/321), done.\u001b[K\n",
"remote: Compressing objects: 100% (207/207), done.\u001b[K\n",
"remote: Total 1643 (delta 208), reused 150 (delta 114), pack-reused 1322\u001b[K\n",
"Receiving objects: 100% (1643/1643), 139.09 MiB | 39.92 MiB/s, done.\n",
"Resolving deltas: 100% (970/970), done.\n",
"Updating files: 100% (179/179), done.\n"
]
}
],
"source": [
"!git clone https://github.com/gilgameshjw/methylprep.git"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[Errno 2] No such file or directory: 'methylprep/docs/example_data'\n",
"/home/jovyan/methylprep/docs\n"
]
}
],
"source": [
"%cd methylprep/docs/"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/home/jovyan/methylprep/docs/example_data\n"
]
}
],
"source": [
"%cd example_data"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Reading IDATs: 100%|██████████| 1/1 [00:01<00:00, 1.76s/it]\n",
"Processing samples: 100%|██████████| 1/1 [00:47<00:00, 47.32s/it]\n"
]
}
],
"source": [
"import methylprep\n",
"from pathlib import Path\n",
"filepath = Path('epic_plus/')\n",
"data_containers = methylprep.run_pipeline(filepath, array_type=None, export=True, manifest_filepath=None, sample_sheet_filepath='epic_plus/samplesheet.csv')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Seems to work without error `AttributeError: 'DataFrame' object has no attribute 'append'`.\n",
"(Note, at one point I thought this would run out of memory, and so don't be surprised if sometimes it does.)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.14"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment