Replies

It was interesting to read that a number of subscribers like to read the (paid) version of the newsletter offline. Since the very beginning of Windows Secrets I have been archiving the e-mails in an Outlook PST file folder, and using one or more e-mail search tools (which have changed over the years) to find things of immediate interest (obviously frequently at a date far removed from the newsletter’s original distribution). This has worked well for me. But now it will serve absolutely no purpose to archive the e-mails, since they no longer contain the full text of the articles. But obviously I could navigate to the web site and view the entire text of all articles for any specific date and then use the web browser’s “Save [As]” facility to save the machine-readable HTML document in a folder somewhere, and then use a different search tool (in the future) to search those files.

However, the process does not work. Note that I said it does not (even) work! Yes, the files are saved (by IE, Chrome, or FireFox), but neither one of those three browsers can (or will) then actually exhibit the saved web page (no matter which web browser was used to save it originally).

In addition, for what used to be a 100 to 200 KB e-mail, the size (on disk) of the saved HTML version of the smallest of the currently-thus-available three newsletters (512, 513, and 514) is 2 MB (to be precise, 2,022,080 bytes), which is an order of magnitude larger! Looking at the saved file, most of it is not actual useful text or images, but JavaScript files and other useless (from an archive perspective) junk.

This is not an improvement — even if it were actually made to work. But, the plain truth is that it simply does not work. Currently, there is apparently no way for a subscriber to archive the machine readable content of any Windows Secrets newsletter. The ten-fold expansion of space required to do so is irritating, but that will probably be a lost cause. Regardless, it simply does not work.

If this BUG is not fixed, I will join the group of lost subscribers. I hope the technical folks at Penton will acknowledge that this is a BUG. Some web sites think the INability to save web pages for future (offline) reference is a FEATURE.

We shall see.

Hi WmHBlair

Well, you certainly made your first post a good one.
That is a cogent and penetrating analysis of what is going on … …
Thanks a lot
Jock

Interesting. I have been hanging out here since the beginning, and reading Windows Secrets and its other now-merged-in newsletters for years, grabbing whatever useful information was available that was not nailed down. All have been very valuable resources for me. Recently, this issue came up again (from an online contact) and I just “knew” I could find some article online that explained what I had already learned at great expense of individual time and personal effort [since I am not a Windows, Linux, or Intel programmer or network administrator — just a so-called “power user”], which turned out to be mostly valuable, as I hinted, for the many other things it occasioned me to be forced to learn. However, I could not find a single post, article, or web page online that expained — nor even hinted at — what I knew. I decided to write one and point my online friend to it. Remembering that one hit out of the literally hundreds that Google enabled (and the situation forced) me to explore was windowssecrets.com, I knew that here is where I had to post it. The audience here is tough. I knew I could be exposed as an idiot if I didn’t know what I was talking about and did not explain it clearly.

But it never even occurred to me that this was the first time I’ve contributed here. Thanks for the compliment. I’ll take it as a challenge, because I’ve been in this same position before here (known the exact answer to someone’s problem), but never took the time to pay the community back for what it and the users here who have taken the time to contribute have given me. As my daughter says: My bad. I promise to do better. So, most of all, thanks for the reminder.

—
WB

> PST is about 5 GB but I have Win 7 64, Xenon processor and 8 GB RAM.

I have suffered from this problem as well, for more than 2 years. Along the way, I developed considerable diagnostic expertise and otherwise useless knowledge of Windows internals in an attempt to gather information (potentially for Microsoft, if I could ever get them to listen to me seriously). Like everybody else who has posted (elsewhere) seemingly authoritatively about what the problem is and what they did to supposedly solve it (or what did not work), I completely missed the boat until I correlated the “not responding” event with what was actually (still) going on in a thread of the Outlook process. I discovered that a highly-fragmented, un-compressed PST file was the cause of my problem. Outlook seems to go out of its way to try to find space in the PST file to store incoming emails (ordinarily a good idea), so that the file size does not grow larger than necessary, but when thousands of emails have been stored and then deleted, this function runs longer and longer, requiring a small amount of processor time but a lot of hard drive activity. The more fragmented (I presume) the PST file is, the longer this search for free space within the currently allocated hard drive file space takes. Other threads are locked while this is occurring, and when this search runs on too long, Windows itself gets upset because it thinks the application is not responding. It’s not, but only because of what is probably a poor design. At that point, the only cure is to terminate the process and allow Outlook to restart itself. Ultimately, that is what you will have to do to recover the application focus, but it’s not necessary to actually do that as soon as Windows thinks its gone to sleep. Outlook is not sleeping, it’s just busy, and working quite well, in fact. You’re never going to talk to it again via the mouse and keyboard (because Windows has decided it’s “not responding”), but under the covers it’s still downloading email messages (albeit quite slowly), searching for free space in the PST file, and storing them. I discovered that if I simply allowed it to “finish” downloading all of the email, it would eventually “finish” having stored all the messages in the PST file. They would not, however, have been run thru the “Rules” engine (because that thread is one that is locked out). But they show up in the main InBox, so all is well. Once Outlook restarts, it “recovers” the PST file adequately (provided I let it “finish”). Repairing the PST file is not necessary; there is, in fact, no “damage” done … unless Outlook is restarted prematurely before all the emails get downloaded.

So the cure was simply to reduce the size of the PST file (in my case to less than 1 GB), at which point the apparent email download speed increased by a factor of at least five. For example, Outlook could download approximately 1,000 email messages in 2.5 minutes, whereas before, with a 4.5 GB PST file, that would take at least 12 minutes, and usually closer to 18 or as much as 25 minutes. All of that extra time was required to drive the hard drive nuts (again, I presume) looking for free space in the PST file that’s already allocated to it (as far as the NTFS file system is concerned).

The way I have Outlook configured now is that the main PST file just receives email and distributes it into folders via Rules. Things I want to keep are then moved to one of several other PST files (whose sizes total around 5 GB in my case). At least monthly I scrub the main PST file for emails to be archived (manually) and then compress the main PST file. At that point, it shrinks from about 1.5 GB of allocated space to about 300 MB. After that, email downloads really speed up. I can now tell from the download speed how fragmented my main PST file is; if I then check the file size, I find it’s grown considerably (by a factor of 4-5 usually). At that point, it’s time to clean it up again, and then compress it. I also then physically move the main PST file, once compressed, off to another hard drive, run Windows drive defrag on that drive, and then move it back; that gets the (now smaller) PST file allocated in one (or at least a small number of) extents, and I think that helps Outlook’s performance when it is looking around within the PST file later as it grows and again becomes fragmented.

Once I started practicing “safe Outlook PST management,” I have never again encountered an Outlook “hang” or “not responding” incident, and I can safely leave Outlook running all day, all night, etc. I went on vacation a month ago and left it running for more than a week. No problems! Before I reduced the (main) PST file size, that would have been a bad idea, since more often than not, simply trying to retrieve a batch of any number of emails (including a small number, like 50) usually caused Outlook to appear to not be responding.

It does not matter that you have a fast processor or a huge hard drive or gobs of memory. Outlook is simply poorly designed to deal efficiently with large PST files, thus resulting in all of the symptoms I have seen reported. No wonder Microsoft was confused. I doubt they tested it with huge PST files under real world conditions. Support for large PST files (larger than 2 GB) is now not a new thing, but I suspect lots of folks are still using the old format (with the 2 GB size limitation), because that is what one gets is one “upgrades” Windows/Outlook (instead of starting from scratch). My experiments indicate that allocating space within a highly fragmented old-format PST file is even more disk I/O intensive than with the new format.

Clean up your main PST file, reduce its size, archive items to be kept to other, less-active PST files, and compress all of them regularly (after backing them up first!) on an appropriate schedule, and I bet your problem will simply disappear. Mine did.

—
WB