Standard Mischief

At some point, I’m supposed to make the obligatory joke about my three readers. Instead, I’m going to share some command line pipes that any fair-to-middle *nix person should be able to figure out. Please note that even though I’m perusing my logs, I’m not doing any creepy crap with the data, and I’m not doing anything funky to readers with cookies or anything. Also, no IP addresses or other personal data were publicly exposed in the making of this blog post.

Although my stats program tells me that I get over 100 unique visitors a day, and it goes as far as breaking them down into spiders vs. “real people”, I’m not entirely sure I’m getting the whole picture. So grep to the rescue.

$ grep 'tailrank.png' access_log|grep '11/Feb'|egrep -o '^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})’|sort|uniq|wc -l

Real people arriving at a web page almost always load available images, while spiders, feed scrapers and hotlinkers usually don’t. So here, I’m going to use one of those below, adorable, social bookmark pictures as a “web bug”, (Wikipedia’s page, and the EFF’s page on them)

The first grep picks only lines that contain the image, the second grep restricts that to only entries for February 11th. The egrep selects only the part of the log entry that has the IP address, and then the list gets sorted, duplicate entries are removed, and at the end, we count the total number of lines. I’m getting 23 people for this date, although once, the IP address numbers were almost identical except for the last digit, so here we can assume that their IP address probably changed during visits that day.

Now for the people who read my feed.

$ grep 'GET /feed/ HTT' access_log|grep '11/Feb'|grep 'Mozilla'|egrep -o '^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})’|sort|uniq


The first grep restricts this to only people asking for my main feed, next restricts to just February 11th again. The third only selects the ones that have “Mozilla” in the user agent string. Then it’s sort and remove dupes again. Here I’ve got six, but I’ve got one overlap with the earlier query, so I’ve got 5 for this day

$ grep 'GET /feed/ HTT' access_log|grep '11/Feb'|grep -v 'Mozilla'|sort|uniq

This one restricts the log to only spiders and scrapers, but gives me the entire user agent string on each one. Bloglines and NewsGatorOnline have the decency of informing me in the user agent string that I have one subscriber with each. The rest are spiders and such, mostly for search engines, and I don’t count them as real.

It’s a snapshot of just one day and does not take in to account of someone who might read my aggregated feed at say Bloglines and also come visit with their browser, nor does it detect someone who might read me both at work and at home, or at two different wi-fi hotspots, but still, I have a pretty good idea of my total readership.

Credit where due: The crude regular expression I used above, (crude, because it would hit on a dotted quad such as 999.999.999.999), came from here.

I got a bug in the system way back when I first installed WordPress 2.0.1

People could leave comments OK, and those comments displayed OK, but when there was a overview of the posts, the comment count was displayed wrong.

This problem followed me through several upgrades. Finally I decided to do a fresh install. Well that didn’t fix things, but I noticed that Spam Karma 2 (hereafter SK2) was no longer bundled with WordPress. I got the latest version and installed it, but I still had problems. At that point I returned to searching my problems in the WordPress forums.

I finally found the problem. There was a big bug in the version of SK2 bundled with WordPress 2.0.1, so the upgrade actually did fix the problem, at least on any future comments. I had Spam Karma 2.0 Final r2 installed. The one you need to upgrade to is currently Spam Karma version 2.2 r3.

To fix older posts you have two known options:

1) Go into the database and do it by hand (yes, that’s slow and you do need to know how to stumble around in mySQLadmin) Specifically wp_posts in the field comment_count category.

or

2) Go to every post that has the wrong comment count, and add a comment. When the new SK2 approves it, the total proper count will be corrected. You could then erase that comment, count should then decrement OK.

Yea, both of those options suck. Sorry, that’s the best I can do. Oh and I really wish that Dr. Dave would splash this all over his home page. I have months of posts that I now need to fix by hand.

Still, to put this in perspective, the only reason comments are open is because of this plug-in. I wouldn’t be blogging without it.

Images from the old blog are still missing, and the template needs fooling with, but things seem to be working OK in the comments department. I’ll have those restored soon.

1. Bid only on items from sellers that are located in your own country.

2. Know the items value. How much is it worth if you bought it through normal retail channels?

3. Only bid on items where the shipping cost is clearly indicated, and deduct that cost from your top bid.

4. Seek out sellers that aren’t pros. They are more likely to undervalue the cost of their own labor.

5. Be careful about bidding on high value items. Usually I find it worth the extra expense to deal with a local merchant, just in case there’s a need to return the item.

6. Paypal sux. It is, however mighty handy for small payments. Don’t get “verified”, don’t associate your Paypal account with any bank account. Fund your Paypal purchases via credit card.

7. Check the seller’s terms to be sure that they are willing to take your unverified Paypal payment. If not, see if they will take a money order or bank check. If it’s a low value item, however, it may not be worth the special trip it takes to get a money order. Buy elsewhere.

8. Use Ebay’s saved serches email or RSS feeds and save yourself some searching.

9. Realize that Ebay will rat you out to the feds merely on their request, no warrant needed. If you buy a CO2 regulator, hydroponic gear and a HID grow light, don’t be surprised if that’s enough evidence for some rubber stamp judge to issue a no-knock warrant for the ninja-fied police to inspect your indoor basil plants and your kegerator.

10. Seek out the misspelled auctions. If you are looking for a good used PanaVise, make sure you search for panavice and ?pana vise? and ?pana vice? too. If no one else finds/bids the auction, you can frequently get stuff dirt cheap.

11. USPS shipment tracking sux. Expect your purchase to be in “Electronic Shipping Info Received” limbo until the item is actually in your mailbox.

12. Expect a few idiots. I’d generally report them but be wary of burning up too much time over a low value purchase. Cut your losses. (That’s the plan at least, I’ve never been cheated. Even Tuan refunded my payment, eventually.)

13. Always leave appropriate positive feedback, but only after you have the item in your hand and you are satisfied. Think twice about leaving negative feedback.

14. Snipe everything. Expect your successful purchase ratio to increase. Expect to win those auctions frequently for less than your top dollar all the time too.

Actually, it’s always the sniping season on Ebay. Since I’m busy gathering materials for a future blog post, I’m reminded again about how useful a program called JbidWatcher really is.

“Sniping” or bidding at the last moment on Ebay has a bad reputation, but I’m not entirely sure why. It’s allowed by Ebay rules. It offers protection from confederates, and it neatly sidesteps all those idiots that get “auction fever”.

Many people who might bid on the same auctions as you won’t bid on anything unless there’s already a bid on the auction. I suppose that’s some sense of herd mentality. Other people see your bid as a challenge, and are willing to bid the auction up, a dollar at a time, to find your high bid. Also, do I need to remind anyone that on the Internet, nobody knows you are a confederate? Confederates are people who act as agents for the seller by placing unscrupulous bids on items to drive up the price. That’s against Ebay rules, but intent is hard to prove unless you can show a pattern.

To sidestep these pitfalls, I just place my bid on the item during the last few moments of the auction. That bid represents exactly my top dollar that I’m willing to pay, and because it’s placed so late no other bidder has time to react to my bid. If course I don’t wait around for the auction to end, I have computers to do boring repetitive tasks like that.

Enter JbidWatcher. It’s a Java based program that will run on almost any platform. It was originally written with the intent of scraping Ebay and monitoring the progress of a seller’s auctions. I believe that “sniping” feature is a requested option, and not within the scope of the original purpose. No matter, it works well, and you can’t beat the price.

OK, so Uncle is talking about emergency car kits again, (earlier, here too), and Jacqueline Passey is complaining talking about no electricity and nearly running out of gas, and I just can’t take it any more.

I think Katrina and the duct tape and plastic sheet alert changed us as a nation, such that when people notice my two-liter bottles filled with tap water tucked away in the corner and ask about them, I can say “Katrina Kit” and they at least don’t seem to get that weirded out look on their faces anymore.

Really though, nothing has changed. When Andrew hit Florida, they had civil unrest and widespread looting, although it wasn’t as severe, and we didn’t have Flicka pools or MMS feeds from local New Orleans TV stations to tell us how bad it was. Back when people half kiddingly asked me about my Y2K prep, I was able to twist the conversation around by reminding them of the bad ice storm we had around here almost exactly a year before. Many people were out of power for three days. I told them that preparing for a week of no power wasn’t too wacknut paranoid.

But this post isn’t about extensive prep for mammoth disasters, it’s about near zero prep stuff you can do before a big snow or ice storm hits, or bad spring thunderstorms, or when the remnants of a hurricane pass over the area.

1. Fill your gas tank.

Jacqueline blames her brother. But if you have any time at all before the storm hits, go fill your tanks. There’s two reasons. First, twelve gallons of gas equals about 72 extra pounds of weight in a car trying to safely pilot around on slippery roads. Second, if everyones loses mains power, no station can sell gas.

2. Fill up some pots and pans with water.

This is still a good idea during a severe winter storm because you can always get frozen pipes, but mainly this is important when hurricane type weather is due. Inevitably during the big storms, the water treatment plant will pass untreated water into the system, and people will be cautioned to boil their water. That can be hard to do when all you have to cook on is an electric stove. I have those refilled two-liter bottles and a few cases of single serve bottles already, but it only takes a few minute of work to fill up a few pans or stockpots.

3. Shut off your water main.

(Yup, this post is pretty long. more below the fold) (more…)