Home Forums Networks & Cross-Platform Networking Downloading text only from web pages

I know there are many ways to do this, but I haven't found one that I'm really happy with.

I often find articles that I would like to download from the web. But I want only the text, not an html file.

I don't want to select the text and then cut and paste. I've done it many times in the past but it is tiresome, a pain in the neck, and often doesn't work very well. I don't want to have to deal with a lot of awkward paragraph spacing, extra space, odd sentence length and so on (Tom Bender's Tex Edit used to do a great job of cleaning up files, but that too was time consuming.) I know the browsers are supposed to be able to download text only files (I have Firefox 3.6 and Safari 4.1) but I can't seem to get it the way I want, text only. I've tried converting html with TexEdit but it doesn't work very well.

This is especially a problem on big files. I tried using TexEdit to convert them, but TexEdit does a poor job of handling long documents. That leaves me with NeoOffice, which should work converting html to text, but it doesn't seem to.

For example, a long file from Project Gutenberg:

http://www.gutenberg.org/files/16350/16350-h/16350-h.htm


2) I also have an xhtml file dowloaded from a Windows desktop but I can't convert it. Surely I have something on my computer that ought to be able to read this file. So far no luck. Tried TextEdit but it didn't work.

Have you tried saving as a PDF file?

Given your OS 10.4.11 and ancient Firefox 3.6.28 I suspect you can't. The latest versions of Firefox have Adobe Acrobat - Create PDF 1.2 loaded as an extension, so it's a piece of cake.

A problem with text conversion on many web pages is because a substantial segment of the text is not HTML, rather it is a GIF, JPG, PDF, or other graphic image. In that case the "text" will have to be processed with with an Optical Character Recognition (OCR) app before it is selectable as text.

For what it is worth, I just downloaded your "long file from the Gutenberg project" and opened it in NeoOffice 2013.1 and it opened with no problems, the text was clear and readable and the links all worked as advertised, as it did in Nisus Wroter Pro 2.0.6 as well (IMO NeoOffice did a better job of rendering and formatting the pages). Pages, on the other hand could not open the file at all. It should be noted I am running OS X 10.8.5, and those versions of NeoOffice and Nisus Writer Pro are probably not available to you in Tiger.

Sometimes there is a "Print" version available for an article or web page; those are usually a lot easier to deal with for a cut-and-paste.

of course, not every article or page will have them, but.

Thanks for the quick replies.

1) Yes, I love when there is a print command giving me a text page with nothing but text. Just doesn't happen often enough.

2) I goofed. I meant I was using TextEdit to try to convert html. Tom Bender's program from long ago was called Tex Edit (he was from Texas).

3) My version of NeoOffice stopped at 3.1.1

4) Maybe a Firefox extension?

I use Text Wrangler from Bare Bones Software to clean up text in cases like this.

If you use Safari, you can often get clean text by hitting the Reader button, then copy-pasting that into a text editor.

Give iCab a try. It can save the files as a text document. Opening them in TextWrangler seems to work better than TextEdit.

If memory serves, the version of Safari in 10.4 does not include Reader.

You might also be able to get a lighter copy by changing your agent-id string in your browser to something that makes your browser look like a mobile device that really ought not to get pushed big flashy banner ads etc.

But in any case, the text is there and in many cases it's not too difficult to sort it out from the rest, but formatting can be lost. Columns that appear side by side will appear one after the other. Tables may be completely split up. Comments, captions, and other unnecessary clutter normally formatted away from the body of the text may appear spliced in with the body.

Are Services available in 10.4? I don't recall. If so, you can select the desired text and then choose Services from the Safari or Firefox menu. At that point, select Print Selection.