« ANN: FeedDemon 3.0 Beta 1a | Main | Making Short URLs Suck Less »

Wednesday, April 01, 2009

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Maybe AV programs all ship with exclusions for the exact filenames Firefox uses, but despite it being sqlite-happy these days I haven't heard about people having problems with the bookmarks/history db, or the cookies db, or the cookie permissions db, or the download history db, or the per-site zoom db, or the form history db, or the web app local storage db, or, or... being locked by AV, which makes me wonder if they all ship ignoring *.sqlite and it's just because they don't know what an .FDB might be that they're constantly getting in your way.

I had the same thought as Phil - I can't believe sqllite is having this problem if it's so popular. What about adding support for .sqlite extension, just during the beta at least, to see if the problem clears up?

Changing *anything* in the anti-virus software is out of the question on IT-managed systems, like here where we are running TrendMicro.

"Update all feeds" is definitely slower than in 2.8RC2.

What about locking the file (and keeping it locked) yourself?

SSDD. We fight the same battles with the same crappy firewalls and AV products. Drives me nuts. A vary large proportion of all support requests end up being related to firewall or AV products.

For the record; I hate them too!

@Phil: I considered the same thing myself, but I haven't seen *.sqlite listed as a default exclusion in popular virus software. A number of other popular programs are affected by this (including Google Gears - see http://code.google.com/p/gears/issues/detail?id=370&redir=1 ), so I'm wondering whether Firefox is doing something to specifically handle this problem?

In Firefox' case, maybe the file access activity is significantly lower than FD's so it's not noticeable? Or maybe there's some caching going on that's held until the file lock clears?

@Mladen: Locking the DB and journal files might be the best option - this appears to be what Firefox does with places.sqlite. I'll give this a shot.

@Jack: Not sure. Firefox's 'places.sqlite' seems to get updated quite frequently. As I noted in my previous comment, though, it looks like Firefox places an exclusive lock on that DB, which may be why it doesn't suffer from the same AV problems I'm seeing.

I'm also using SQLite in one of my product and have the same problem. I ended up with encapsulating all database access in methods that will wait for a little while and retry if they fail with certain errors. Not ideal, but seems to work OK.

Please let us know if your solution with locking the file(s) helps!

Threads are your friend - that is unless the anti virus app actually suspends your app while it does it's evil deeds. (I hate you Cisco Security Agent)

Suggestions:
* Change the file open flags (share for reading) when you open your DB file
* move all your DB file access into it's own thread
* move your DB file to an unnamed memory mapped file and then in a worker thread, write it out to disk when it's data changed.

@Brian: I added "PRAGMA locking_mode=EXCLUSIVE" to the latest beta, and it appears to have made a difference. I have one error report that shows a SQLITE_CANTOPEN error, whereas I have dozens of SQLITE_CANTOPEN errors from the previous build (and that one error may be a red herring).

Good suggestions, Jason. However, moving all DB access to its own thread adds a level of complexity (esp. with regards to UI updates) that would most likely cause more problems that it's worth.

The memory-mapped file idea could work with non-critical data, but since an app/OS crash could mean that data is never written to the DB, I wouldn't trust it for critical data. I'd rather have a handful of problems with AV software than hundreds caused by OS instability :)

@Brian: BTW, something I did in previous builds that seemed to help was to use "PRAGMA journal_mode=PERSIST" to tell SQLite to reuse the transaction journal file. This way AV software doesn't see a new file being created with each SQLite transaction, giving it less reason to get in the way.

Hi Nick,

thank you for the tips, I'm going to give them a try!

Best Regards,
Brian


I got Cyberdefender, and it worked great as a free scanner with spyware and trojan removal. If it finds a virus, you need the upgrade, which I got since I liked the speed and user interface of the scanner. The Cyberdefender anti-virus works great, and the paid version I got also came with 24/7 computer help line with my wife found helpful while I was away. I found out Cyberdefender is a NASDAQ campany and they have a great product.

The comments to this entry are closed.