Sunday, June 30, 2013

A Bukkit Jython Console Plugin for Minecraft

Summary

This post describes how you can create a Python interpreter for a Bukkit Minecraft server, providing access to the full internal API. Commands are issued on a separate console window and do not require players or server administrators to be running the game itself.

Introduction

Unless you've been living under a rock, you probably know about Minecraft, the famous open block based 3D building/crafting/exploration/survival game.
Minecraft is a game about breaking and placing blocks. At first, people built structures to protect against nocturnal monsters, but as the game grew players worked together to create wonderful, imaginative things.

It can also be about adventuring with friends or watching the sun rise over a blocky ocean. It’s pretty. Brave players battle terrible things in The Nether, which is more scary than pretty. You can also visit a land of mushrooms if it sounds more like your cup of tea.
Minecraft has been wildly successful. Not only because of the LEGO-like main gameplay, but also thanks to the developers' friendly stance towards modding, causing an incredible amount of modifications being released, ranging from additional creatures to custom biomes.

Lots of people have also emphasized the fact that Minecraft is a great way for parents to bond with their children. Also in the classroom, Minecraft has been used as a teaching aid to inspire children.

Combine the two aspects above (modding and teaching), and you ultimately start wondering whether Minecraft could also be used to teach programming. When the Raspberry Pi put online their announcement that a special Minecraft version would be available for their miniature, cheap computer, the answer to this question indeed seems to be: yes, since their "Minecraft: Pi Edition" would come with a programming Python API built in, as shown by the following screen shot.

Minecraft running on the Raspberry Pi

Back on the PC-side, the great interest in modding Minecraft eventually led to the Bukkit project -- a community-created Minecraft server with a comprehensive API to allow others to create plugins, extending the world, gameplay and rules of the vanilla Minecraft experience. Minecraft server administrators also enjoy having some sort of console available to be able to modify a Minecraft world at runtime. Banning players, erecting or destroying structures or changing the time of day without having to restart indeed comes in handy when you're running a server. The Bukkit development API makes it very easy to create such kinds of plugins, but they often require the list of possible commands to be specified beforehand (in the Java plugin), which are then made available through the game's chat window. For instance, issuing the command:

/weather sunny

Would mean that you have a plugin running which is able to handle the command /weather and converts this to appropriate Bukkit API calls, e.g. setThundering.

Other than that, lots of people have started to learn Java or other languages by trying to create a Bukkit plugin or some sort of Minecraft-inspired clone. Following list provides just a tad of what is out there:
This is all nice, but a bit too high-level for what I wanted to build. I wanted to have direct access to the Bukkit API, similar to how the Minecraft: Pi Edition provides direct access to its API.

We're thus going to create a Bukkit plugin which gives server administrators or programming students direct access to the full Bukkit API. The actual plugin is actually very simple, thanks to the magic of Jython, a Python implementation in Java which allows for Python<->Java crosstalk and saves us from having to implement any command in the plugin itself. Commands are issued on a separate console window and do not require players or server administrators to be running the game itself.
The base idea behind what we're going to do is certainly not new by any means. The Minecraft: Pi Edition, as mentioned above, implements a similar idea (see also this article). The Mag Pi magazine even featured a nice tutorial to show off what you can do with it.

Other than that, there's ScriptCraft, providing a Javascript based programming interface for Minecraft:
ScriptCraft is a Minecraft Mod that lets you extend Minecraft using the Javascript Programming Language. ScriptCraft makes modding Minecraft easier. It includes a logo-like "Drone" object that can be used to build complex buildings, roads, villages, even entire cities. It also includes many other features that make modding Minecraft easier.
You can find more information about ScriptCraft here and here. It's certainly a great project to teach programming, but comes with the drawback that it offers programmers its own API (implemented in a Bukkit plugin), meaning that you'll have to program using Bukkit chat commands, for instance:

/js box("5").up(3).left(4).box("1").turn(3).fwd(5).right().box("1").move("start")

ScriptCraft also includes a "Drone" class which simplifies movement and building in Minecraft, similar to the "turtle" in the LOGO programming language (LOGO is a Lisp-like programming language which used to be popular to teach kids programming; fun fact: the first "real" programming language I learnt actually was a LOGO derivative).

This is all nice, but still too high-level. I want to be able to get direct access to all Bukkit's API functions (not only exposed ones through a plugin), preferably not having to use the game's chat.

ComputerCraft also inspires people to learn coding, but is even more high level. It actually provides a "console block" in the game offering a rudimentary shell which has nothing to do with the inner workings of either Bukkit or Minecraft.

Finally, plugins such as PyDevTool and RedstoneTorch come closer to what we want to achieve. For instance, PyDevTools allows to:
Execute arbitrary python code from console or ingame
Execute saved python scripts
Interactive interpreter mode
This is almost exactly what I wanted to build, but still requires to log into the game to execute /py <statement> commands. Yes, these can also be entered in the Bukkit console (offering some kind of interactive shell), but I wanted to remove the /py clutter. A simple trick indeed will allow us to do so, let's get right to it...

Making the Plugin

We'll be using Eclipse to make our Bukkit plugin. I'll assume you know how this IDE works already. If you're not familiar with Java and just want to try the plugin, you can skip this section. Otherwise, follow along.

First of all, we're going to create a new Java project in Eclipse. I'll call this BukkitConsole. We create a plugin.yml file in the project root with some basic information:


Next up, we create a lib folder in which we're going to place two libraries:
  • jython-standalone-*.jar: to be downloaded from the Jython web site
  • bukkit-*.jar: the Bukkit development API, which can be downloaded from here (more information on their Wiki)
Add these libraries to the build path. You'll now have the following structure:


Next, we'll create the plugin classes themselves. Create a package under src called com.macuyiko.bukkitconsole. We're going to add two classes.

MainPlugin.java, this class just loads the libraries from the lib folder and spawns a PythonConsole window:

package com.macuyiko.bukkitconsole;

import java.io.File;
import org.bukkit.plugin.java.JavaPlugin;
import org.bukkit.plugin.java.PluginClassLoader;

public class MainPlugin extends JavaPlugin {
    public void onEnable(){
        getLogger().info("BukkitConsole: Loading libs");
        try {
            File dependencyDirectory = new File("lib/");
            File[] files = dependencyDirectory.listFiles();
            getLogger().info("BukkitConsole: JARs found: "+files.length);
            for (int i = 0; i < files.length; i++) {
                if (files[i].getName().endsWith(".jar")) {
                    getLogger().info("BukkitConsole:  - "+files[i].getName());
                    ((PluginClassLoader) this.getClassLoader()).addURL(
                        new File("lib/"+files[i].getName()).toURI().toURL());
                }
            }
        } catch (Exception e) { 
            e.printStackTrace();
        }

        getLogger().info("BukkitConsole: Starting Jython console");
        new PythonConsole();
    }

    public void onDisable(){
        getLogger().info("BukkitConsole: Plugin was disabled");
    }
}

PythonConsole.java, this class employs Jython to execute console.py in the python directory:

package com.macuyiko.bukkitconsole;

import org.python.core.Py;
import org.python.core.PyString;
import org.python.core.PySystemState;
import org.python.util.PythonInterpreter;

public class PythonConsole {
    public PythonConsole() {
        PySystemState.initialize();

        PythonInterpreter interp = new PythonInterpreter(null,
                new PySystemState());

        PySystemState sys = Py.getSystemState();
        sys.path.append(new PyString("."));
        sys.path.append(new PyString("python/"));

        String scriptname = "python/console.py";
        interp.execfile(scriptname);
    }
}

Things'll now look as follows:


That's it for the Java side! Export your project to bukkitconsole.jar.

We're now going to setup the server. Create a folder server somewhere. Next, download craftbukkit-*.jar from here (Bukkit is the development API, "CraftBukkit" is the actual server; again, more info on the Wiki) and put it in this folder.

We're also going to create a simple run.bat in this folder:

java -Xmx1024M -jar craftbukkit-*.jar -o true
PAUSE

Execute run.bat to see whether CraftBukkit works fine and if you can connect with your Minecraft client (note: if you're running the current snapshot, the launcher should allow you to create a profile with version 1.5.2, which is what we're using here). Normally, you should get something like:


Enter stop to stop the server. CraftBukkit will have create some new folders in your server directory:


Next, we're going to download a Jython console. I picked this one by Don Coleman, since it is simple and offers fairly robust code completion. Note that I had to make some changes to it which can be found on the GitHub repository I've created for this project. Not using these changes is okay but will lead to some error spam on the Bukkit console.

Create a folder called python in server and extract the Jython console. python should contain:
  • console.py
  • history.py
  • introspect.py
  • jintrospect.py
  • popup.py
  • tip.py
We also need to create a lib folder in server and put in the following JARs:
  • bukkit-*.jar
  • jython-standalone-*.jar
Yes... the same ones we've included in our Eclipse project. It would have been possible to package these into the JAR, but loading in resources from JAR files is a bit of a pain which I didn't want to go through. Feel free to add lib and python to the JAR and submit a pull request on GitHub if you get things working.

That's it. Try executing run.bat again. If all goes right, Bukkit should be able to load in our console plugin and will present you with an interpreter:


We can try out some commands. Note that you'll have to from org.bukkit.Bukkit import * to get access to the Bukkit API (or from org.bukkit import Bukkit -- you'll have to use Bukkit.* but will allow autocompletion)!

>>> from org.bukkit.Bukkit import *
>>> getWorlds()
[CraftWorld{name=world}, CraftWorld{name=world_nether}, CraftWorld{name=world_the_end}]
>>> w = getWorld("world")
>>> w.getTemperature(0,0)
0.800000011920929
>>> w.getBiome(0,0)
PLAINS
>>> w.getTime()
12557L
>>> w.getPlayers()
[]
>>>

The Bukkit API docs provide a full overview of all that is possible with the complete API. Broadcasting messages, changing inventories, changing worlds, spawning lightning, all is possible.

You can also save functions in Python scripts if you want to. For example, try creating a scripts.py in the server folder:

import random
import time
from org.bukkit.Bukkit import *

def punishPlayer(name, power):
    p = getPlayer(name)
    p.getWorld().strikeLightning(p.getLocation())
    p.getWorld().createExplosion(p.getLocation(), power)
    broadcastMessage("Player %s has been punished!" % (name))

Now, in the console, we can:

>>> import scripts
>>> scripts.punishPlayer("Macuyiko", 6)

Which will explode a player with a lightning strike to warn other outlaws. The server administrator's "command center" now looks like this:


Wrapping Up

That's it. We've created a Bukkit plugin spawning a Jython interactive console which is able to use the full Bukkit API.

This is a nice starting point to create administrative support scripts, in-game games, or course material for a Minecraft Python programming course.

The following video shows some more tinkering:



(Protip: I'm a bit awkwardly using alt-tab and the inventory screen here. If you press T to start typing in chat, you can alt-tab out of the game without pausing or having the inventory screen opened, which allows you to better see what's going on.)

The source code for this project can be found on the following GitHub repo. The repository also hosts everything you need to get running without having to code/compile anything. Feel free to modify both the Python and Java sources and submit pull requests with improvements. I'd also love to hear about any cool or interesting things you do with this.

Saturday, June 29, 2013

I Almost Switched to Pelican, a Static Blog Engine

Visitors familiar with my blog will notice a pretty big change when viewing this post. The blog's been due for a redesign for some time now, so a few days ago I finally went through with it.

The redesign took more time than expected, since I spent quite a lot of time setting up Pelican -- a static blog engine written in Python, including converting all my Blogger posts to Markdown. I will describe some of the reasoning behind this, together with the reason why I ultimately stayed with Blogger -- for now.

What's Wrong with Blogger?

I've been using Blogger for many years. Delving into the archives of this site suggest I've been seriously using it since 2005, with my main blog being called "Macuyiko's Mindcube" back then. In fact, it's still online, with the last post redirecting visitors to my new blog ("Bed Against The Wall").

Throughout the years, that -- this -- blog went through a series of redesigns and shifted through topics of interest. To be honest, I was never that good at consistently putting out content, but being able to keep the thing running for almost ten years now is an accomplishment in itself, I suppose.

Anyway, I've been getting somewhat disappointment with Blogger lately, something which started some two or three years ago, when I started putting out longer articles, containing also more code fragments. Don't get me wrong, I don't think Blogger is bad, but the following things were becoming an annoyance:
  • I've seen my fair share of Javascript WYSIWYG editors -- either bolted to your favorite blogging platform of choice or "that PHP Content Management System from the nineties". While libraries and frameworks such as JQuery have helped to clear out some of the mess, I honestly feel that 99% of them still behave like crap.
    I'll not go into details about all the things wrong with Javascript editors now. Still, there is one silver lining which is worth mentioning: the amount of great Markdown based Javascript editors which have appeared during the past years. For example, check out EpicEditor, markItUp!, markdown live editor, WMD (outdated) and its many spinoffs. I absolutely love Markdown, and have even forced it to users of websites I've built, using one of the aforementioned Javascript editors to make it clear how easy it is to write Markdown text.
    At this point you might be wondering what, exactly, is so messed up about the Blogger WYSIWYG editor. There isn't anything really wrong with it, but the following has been bothering me:
    • Inserting code is a problem. You have to deal with pre, tt, code tags together with installations of Pretty Print (or similar libraries). Don't forget to convert your <'s to &lt; or things won't work. Make sure your whitespace is correct, meaning that you alternate between tabs, spaces, or a bunch of &nbsp;'s.
    • So okay, maybe Blogger isn't great for code, but at least it's perfect for writing, right? Not really. It's okay, but the rise of recent "modern magazines" such as Medium and SVBTLE have made it clear that writing can be a much smoother process. Heck, even Wordpress' editor is nicer, and will apply many typographic fixes such as converting two dashes to an em-dash.
    • Blogger allows too many formatting options. Since its editor is a WYSIWYG on top of HTML, it's perfectly fine to copy-paste in ugly text coming from an e-mail message or that Word document you typed out... <span> with mso-style, anyone? Please, just allow me to apply headings, emphasis, lists and paragraphs to my text, but let the theme CSS do its thing and leave out the styling.
  • Making changes to the template is bothersome. Blogger's template system is fine as long as you keep using one of the default themes, maybe make some slight color or font changes, and leave the rest as is. Once you want to start moving stuff around however, add backgrounds or lots of custom CSS/JS, things get harder and you quickly and up in the middle of "tag soup". I didn't even manage to change the background of the final theme I was using once, even though I was initially planning to do so once in awhile. It's not hard, but my laziness outshines the difficulty.
That said, there are plenty of things which I do like about Blogger. It's very easy to set up a basic blog, there are many widgets to choose from, basic templating stuff is fine if you don't mind your blog having a common look and feel. Also, comment management and moderation works pretty well, especially now with the Google+ integration.

So, in short, what I wanted was:
  • The ability to write in Markdown
  • The ability to easy add in code, syntax highlighted if possible
  • Nice typographic options, beautiful text
  • Easy theming support
Blogger was starting to make this pretty hard.

Solutions, Workarounds and Alternatives

Just continue to use Blogger?

I'm not alone in my desire to use Markdown with Blogger. The topic has actually been discussed to a great extent, with solutions including the following:
  • Write Markdown on your computer, then convert to HTML with one of many tools out their. Cons: redundancy, no central "post" repository, making edits is hard. Not 100% web based.
  • Use one of these neat apps such as Byword (MarsEdit as well?) which lets you publish to Blogger. Cons: not 100% web based, no central posts repository. Often Mac only...
  • Write a web app using Blogger's API which would allow me write posts using Markdown and publish them to Blogger after converting them to HTML. But this would've taken so much effort.
  • Include a bunch of magic Javascript scripts/parsers on your Blogger blog which will perform the conversion of Markdown to HTML client-side. This keeps your posts centrally stored, but relies on too much hacks to feel like a great solution. Still, it's being used. mmd.js seems to be the preferred Javascript Markdown parser as of now.

Switch to another service

There has been a renaissance of writing on the web. When 2012 used to be about Tweets and short status updates, sites like Neocities or Medium are putting the emphasis back on simple, well written web sites.
Indeed, the options for a minimalistic blog are many. Many of them support Markdown out of the box. Many of them do away with many typical overblown blogging features. No comments, no complex theming, just writing:
I will admit, these all look fantastic. I'm already using Tumblr for my random stuff I find online blog; the Dropbox based options look like great Zen-like blogging experience. I have my invite for Medium, and the platform looks fantastic.

I would've gone with one of these options (in fact, I'm still on the fence), but I was a bit disappointed with (i) the lack of code formatting support on most of these platforms, (ii) the lack of any theming to make yourself stand out.

Host it yourself, static-like

A final alternative solution I was considering was hosting the blog myself using a static blog engine. I don't mind Wordpress, but think it's completely overkill for a simple use case such as mine. I considered the following options:
  • Jekyll: Ruby, Markdown or Textile, robust, proven, powers GitHub Pages...
  • Octopress: sits on top of Jekyll to provide a more full featured blogging platform.
  • Hyde: "Jekyll's evil twin". Python, Jinja2, full-featured.
  • Hakyll: Haskell, robust, esoteric.
  • Acrylamid: Python, Markdown and reStructuredText, Jinja2. Fast, experimental.
  • Utterson: Unix, HTML. Basic, experimental.
  • Rog: Ruby, simple, young.
  • Cyrax: Python, Jinja2, young.
  • Second Crack: PHP, one of the few PHP static blogging engines worth mentioning these days (shows how the language has evolved, actually...). Created by Marco Arment, so you know it can hold its own. Not suitable for everyone though.
  • Pelican: Python, Jinja2, Markdown and others, well-featured, easily usable.
Ultimately, I settled on Pelican. I liked Jekyll, but wanted a Python based engine since I didn't want to deal with installing Ruby on the server I'd be using. Installing Pelican is very simple. Just pip install pelican, Markdown and typogrify and you're set. Typogrify is a great Python library which helps you to prettify written text, mostly thanks to Gruber's fantastic SmartyPants plug-in.

Configuration is just an easy step consisting of modifying pelicanconf.py:
#!/usr/bin/env python
# -*- coding: utf-8 -*- #
from __future__ import unicode_literals

AUTHOR = u'Your Name Here'
SITENAME = u'Your Title Here'
SITEURL = ''

TIMEZONE = 'Europe/Brussels'
DEFAULT_LANG = u'en'
TYPOGRIFY = True

# Generation
DIRECT_TEMPLATES = ('index', 'archives')
MONTH_ARCHIVE_SAVE_AS = 'archives/{date:%Y}/{date:%m}/index.html'
YEAR_ARCHIVE_SAVE_AS = 'archives/{date:%Y}/index.html'

# Feeds
FEED_ATOM = 'feeds/atom.xml'
FEED_RSS = 'feeds/rss.xml'
FEED_ALL_ATOM = None
CATEGORY_FEED_ATOM = None
TRANSLATION_FEED_ATOM = None

# Blogroll
LINKS =  (('Name', 'http://url.com/'),
    )

# Social widget
SOCIAL = (('Twitter', 'http://twitter.com/username'),
     )

DEFAULT_PAGINATION = 10

# Template
THEME = "themes/custom_theme"

I then picked the TuxLite ZF theme from pelican-themes as a starting point to put in custom_theme. I changed the color scheme, header sizes and spacings and fonts (making use of Google's web fonts to pick out a nice heading font which would work well with large-sized OpenSans for the normal text, which I'm currently in love with -- it works great for slides as well, try it!). Editing the Jinja2 themes was a breeze and mostly consisted of taking stuff away -- like tags and categories -- which I didn't need. Finally, I made sure to made the background "pop" behind the single-column content, using inspirational, high resolution foto's.

Next up, the hard part, converting all my Blogger posts to Markdown... Luckily, Lars Kellogg-Stedman has had the same problem and describes possible approaches in this post. I thus created a folder with the following:
You'll need to install lxml to get everything to work. This didn't work for me using pip, but easy_install did the job. Once this is setup, create a new directory posts and run:

blogger2md.py --html2text --output-dir posts blogger-export.xml

The script will start crunching and convert everything. Afterwards, I went through every post to rename the file name, correct the meta data, fix the links (Pelican wants (|filename|FILE_PATH)) and clean up some formatting issues still present after the conversion (quite a lot of them). This took a few evenings, but I now have a collection of Markdown posts for this complete blog.
Side track: a trip down memory-lane
While I was converting all these posts, it struck me how long I've been blogging, and how "face-palmingly" bad my writings were as a kid. Luckily, my Geocities page has since long disappeared thanks to Yahoo! (actually, I should retrieve it from the archive for a laugh), but this wasn't that much better. "This dog is gonna check out town"... really, young me? I wanted to keep everything intact, but made some modifications to the language where I really felt I had to (swearing, "hackz0r M$"-bashing, referenced to piracy... hey, I was a poor kid).

It's also funny to see the topics my blog has shifted through:
  • Games, with sad reviews of "classics" such as Scarface... and Portal.
  • Linkdumps, oftentimes based on what I'd read on Digg (yes, Digg!) that day. Is Digg still around? I almost forgot about the HD-DVD "number gate" back then.
  • Linux related posts. Mostly following after acquiring my now pre-historic Thinkpad X60 which is, in fact, still somewhat being used.
  • Hacking and thinkering... like here and here. Some of these are not that bad. The Kippo article has been pretty popular, together with the PalettestealerSuspender tool. Others just show off my horrific PHP skills.
  • Puzzles, algorithms, optimization. I like these as well, but they should be proofed for language, spelling, formatting.
Anyway, back to the main programming now.

Just stay with Blogger?

So here I was, ready to launch with Pelican. Admittedly, some of the posts needed some further formatting, some links has to be corrected and the Disqus comment system had to be implemented, but that could be done later. All there was left to do was relink the domain name (blog.macuyiko.com) and put up a message on the old blog that things had moved.

Still, I wasn't feeling sure about things. Just to be sure, I decided to have another look around to see if things had improved on the Blogger side as of yet, since it'd been a while since I'd searched for solutions.

Turns out there is a web app called StackEdit which provides a great Markdown writing environment, is able to sync to Dropbox or Google Drive (sweet) and is able to publish to Blogger. Alright, maybe one more chance, then. I spent some time revamping the old theme to match the design I'd put together for the static site (you're probably looking at it now), using the same fonts and colors. I had also set out to be able to change the background every time I made a new post, so I made sure I'd be able to do that from now on as well. That's why you're currently looking at a summer-y "pelican" background. It'd be great if I could made the background post-dependent, but maybe that's going too far.

So I'm keeping things a bit longer with Blogger. As I said at the beginning, it offers some nice features, and this way I don't have to introduce yet another commenting system.

On the plus side, I'm now able to write and keep posts as Markdown (using StackEdit with Google Drive) and host at Blogger. The theme doesn't look too bad (will need some further fine tuning) either. Still, if it turns out that I'd rather go to Pelican anyway (or maybe one of the services mentioned above), I now have a collection of Markdown posts at the ready.

I realize this post didn't really offer any information. I guess this was an exercise in keeping things the same while still putting in lots of work. Still, it was nice reading up about static blog engines, and I'm still pretty intrigued by these hip, new services (Blogger certainly feels like the old guy in the block). In addition, the refresh of the Blogger theme has been inspiring in trying to conjure up some new content, so the remainder of the year will see the arrival of some other posts as well -- so long as this experiment goes right, this is a first test post after all. I should still go through the older posts to clean them up, but we'll see what we can manage.