Can't Open .ipynb file in Jupyter: Too Large

[WWGD]

Hi All,
I may have asked this question earlier. Apologies if I did.

I have a Python .ipynb file in a Jupyter Notebook whose size is 99.3 MB. Thing is most of it is not code:
I wrote some code; specifically a loop incorrectly and it iterated way too many times. I saved the file, which included this repeated loop. Now I just get "File failed to load" error messages. Someone suggested ( it may have been here in PF) to change the extension to some text file, open it and delete the repeated output. But I don't see how to do this .
Any ideas?

 

[Borg]

Change the .ipynb extension to .txt and open it with notepad++.

 

[WWGD]

Change the .ipynb extension to .txt and item or with notepad++.

Thank you, I thought of that, but I am not sure of how to do it. The format of the notebook does not seem to allow for that. These files seem to be stored in the back end server of Jupyter. Edit: And are not accessible from the Windows file manager.Maybe I can use save as and store them in a path. using the .txt extension. Will give it a try.

 

[Borg]

Sorry about the typo earlier (phone autocorrect). Notepad++ should be able to handle a 90Mb file without too much difficulty. BTW, you don't really have to even change the extension - just open it directly with Notepad++.

Once you open it, the notebook is just a JSON file when you open it. Notepad++ will show the line numbers on the left side and should highlight an entire section when you click in it. Just look for one that's really big. When you remove the extra junk, you'll have to make sure that it's still well-formed json when you're done - no dangling brackets like this { that don't have the ending bracket (or vice versa). I would advise two things with this:

• Practice on manually removing content from a smaller file first to make sure that it opens when you're finished.
• Make a backup of the file that you're trying to fix and only do this to copies of the original.

 

[Mark44]

I have a Python .ipynb file in a Jupyter Notebook whose size is 99.3 MB. Thing is most of it is not code:
I wrote some code; specifically a loop incorrectly and it iterated way too many times. I saved the file, which included this repeated loop.

Why do you want to open this file? Searching manually through a 99.3 MB seems like looking for a needle in an array of haystacks.

 

[pbuk]

nbstripout was made for this task.

If you want to do it manually:

Thank you, I thought of that, but I am not sure of how to do it. The format of the notebook does not seem to allow for that. These files seem to be stored in the back end server of Jupyter.

No, the 'back end server of Jupyter' just stores them in the file system, wherever you tell it to, defaulting to your home directory which on Windows is something like C:\Users\WWGD. Don't you recognise the path when you browse files from the Jupyter window?

Edit: And are not accessible from the Windows file manager.

Either you are looking in the wrong place or you have some daft setting in File Manager. If it is the first, type

cd C:\
dir *.ipynb /S

and wait to find all the notebooks on your C: drive (and repeat for other drives if necessary).

Maybe I can use save as and store them in a path. using the .txt extension. Will give it a try.

Unless you have a Jupyter plugin for Notepad++ it would be better to change the extension to .json, then you should get code folding which will help you. If you do have a Jupyter plugin then just leave the extension.

 

[pbuk]

Why do you want to open this file? Searching manually through a 99.3 MB seems like looking for a needle in an array of haystacks.

Unless the file is corrupted there is no need to search manually, any code folding editor will quickly separate the wheat from the chaff.

 

[WWGD]

Why do you want to open this file? Searching manually through a 99.3 MB seems like looking for a needle in an array of haystacks.

A Ctrl +F should help.

 

[WWGD]

Now, I found the file path ( Indeed in C:\Users\...) I've tried to open the file ( With a .pdf extension) , in Thonny and Thonny shuts down each time ( 3 times so far).

 

[pbuk]

Now, I found the file path ( Indeed in C:\Users\...) I've tried to open the file ( With a .pdf extension)

If it has a pdf extension it is probably a pdf file, not a Jupyter notebook.

in Thonny and Thonny shuts down each time ( 3 times so far).

Ditch Thonny, you need a proper code editor. For most purposes I recomment Visual Studio Code, but it barfs on really big files so for this purpose on Windows I would use Notepad++. But it won't be any use for a pdf file, that is not going to contain your code.

 

[WWGD]

If it has a pdf extension it is probably a pdf file, not a Jupyter notebook.


Ditch Thonny, you need a proper code editor. For most purposes I recomment Visual Studio Code, but it barfs on really big files so for this purpose on Windows I would use Notepad++. But it won't be any use for a pdf file, that is not going to contain your code.

Thanks; will try resaving as a .ipynb and see if I can open it.

 

[pbuk]

This short script will strip the outputs from a Jupyter notebook (and will run as a Jupyter notebook itself).

import json

# Enter your filename here.
filename = 'your-file-name-without-extension'

ext = '.ipynb'
outfilename = filename + '-stripped'

# Parse the notebook file into a dict and reference its 'cells'.
infile = open(filename + ext)
notebook = json.load(infile)
infile.close()

cells = notebook['cells']

print('Removing output from', len(cells), 'cells')
for cell in cells:
    if 'outputs' in cell:
        cell['outputs'] = []

# Dump the modified dict as prettyprinted json (= ipynb).
print('Writing to', outfilename + ext)
outfile = open(outfilename + ext, 'w')
json.dump(obj, outfile, indent = 2)
outfile.close()

 

[WWGD]

Thanks, @pbuk , just a follow-up. It worked well. Thanks again.