GNU Hurd/ community/ weblogs/ ArneBab/ 2011-04-06-application-pyhurd

Welcome to... ... the GNU Hurd!

Python Bindings for the Hurd (PyHurd)

Contact information

  • Name: Arne Babenhauserheide
  • E-Mail Address: [email protected]
  • IRC-nick: ArneBab @ freenode
  • Jabber-ID: [email protected]
  • Phone-number: XXXXXXXXX
  • GnuPG key: http://draketo.de/inhalt/ich/pubkey.txt

Who I am

I am a physics student from Heidelberg, Germany, a passionate free software user and roleplayer, and I started contributing to the Hurd in minor ways about 5 years ago. Now my coding skills are good enough (and I have enough time) that I feel ready to tackle a GSoC project - and I want to take the chance GSoC offers and do a focussed effort for contributing to free software before I am no longer a student. I married 4 years ago and now have a 5½ month old son whoose happy laughing can make you forget everything around you - or at least it does that to me, but what else could you expect to hear from his father about him ;)

Project

For this years GSoC I want to turn the currently rudimentary Python Bindings of the Hurd into a complete Python-library for low-level Hurd and Mach hacking with high level functionality to allow for easy creation of complex applications. Particularly it should make it possible to utilize the whole Python standard library for translators.

Preliminary Schedule

  • Community bonding period. Read up on the current C-interface to the Hurd and Cython. Especially grok the Hurd hacking guide. Add docstrings to all existing source files (where they are missing) explaining what they do. Add auto-generated API-docs. Deliverable: Easy to understand API-docs of the current PyHurd, a simple way to generate them from the sources automatically.
  • May 23. Coding starts.
  • May 30. Finished a basic Hello World translator, naively implementing the necessary Mach parts directly in the translator.
    1. A simple program which gets a Mach port and can receive messages on that port. It has to get and hold its port at startup and send a reply port, needs to use mach_msg to get the messages, should be able to deallocate the port and must have a kill condition (for example 10 received messages).
    2. stdout functionality, to print all Mach messages (for debugging and to make sure that they really get received entirely).
    3. a parser for the Mach read file message similar to trivfs_S_io_read

  • June 6. Moved the functionality for reading into a simple API using decorators to define actions and ported Hello World to use it:

    """Show Hello World."""
    from translator import repres
    @repres.text.event
    def on_text_read(size):
    return "Hello, World!"[:size]

  • June 13. Implemented single file read-write in the API. Added a simple writethrough translator. The API code is nicely commented.
  • June 20. Access Control and file attributes. Added lock_file translator which just adjusts the effective file access codes and can be used to lock a file as long as the translator is used. Might be useful for quick testing.
  • June 27. Translator commandline arguments and testing.
  • July 4. Translator: Overlay with backend store: write changes to a different file. Makes any file writeable, but keeps the changes only visible for the user who set up the translator. Effectively single-file unionmount.
  • July 11. Mid-term: trivfs in python works: It is possible to write translators in Python with relative ease.
  • July 18. More complex, specialized and helper translator libraries, along with example translators. This should recreate some of the Hurd libraries for Python and add convenience options.
  • July 25. Full featured setttrans in Python.
  • August 1. Redesigned and realized an updated controlling API with the existing direct Cython bindings.
  • August 8. More translators and integrating into the build system.
  • August 15. Suggested Pencils down. The translator API is easy to use, there are many example translators and there is a full featured settrans command in Python using the easier controlling API which shows how to control the Hurd directly from Python. The code is pushed to https://github.com/ArneBab/PyHurd and a git repo at https://git.savannah.gnu.org/cgit/hurd and integrated into the build system with a switch to enable building PyHurd.
  • August 22. Firm pencils down.

Initial Fix

Initial Fix: Making PyHurd build again under Cython 0.14.1. Sent as patch series to [email protected]

Detailed answers

What I have to learn, and what I already know

I need to dive into the detailed interfaces of the Hurd to get a better understanding of the exact requirements for a well usable Python interface, especially for higher level functionality, and read up more on working with Cython.

I already know Python and I did design my share of interfaces for my own hobby projects (TextRPG, Fungus, evolve-keyboard-layout and others).

Also I know the functionality and design of the Hurd from a user perspective and can code in C and C++.

Why did you choose this project idea? What do you consider most appealing about it?

FIrstoff: It is about making it possible for me to hack on the Hurd using my favorite programming language.

Also I can learn more about accessing low-level interfaces directly (as opposed to just using higher level abstractions) and grok the ins and outs of creating Python extensions - into which I wanted to dive for a long time now.

And I helped getting the project running and am intrigued by how far it can be pushed.

Have you been involved in any free software ("Open Source") projects yet? Which projects, how long, and in what way have you been involved? Have you been active in the Hurd project/Hurd community before?

I worked on documentation and news for the Hurd, wrote two plugins and the usage guide for Mercurial and created a bunch of personal Python projects. Also I generally try to nudge other Hurd developers into the direction of actually getting the system useful for people (and communicating its strengths) - and do the same for the freenet project.

In my opinion, my major contribution to the Hurd is the Month of the Hurd, a try at fixing the Hurds reputation for never being finished. To achieve that goal, the Month of the Hurd only lists actually testable successes for which I can easily describe how they get the Hurd closer to its vision, ideally those which are already committed.

Please briefly describe the Hurd, including the goals, architecture etc. Also, what makes you interested in the Hurd? Why do you want to work on it? What is your vision of it's future development?

The Hurd offers much greater freedom for users compared to Linux, because every user can change his/her environment to a much greater extent.

Also it allows for easier low-level tinkering, making it possible for hobby-hackers to work on stuff which in linux requires dabbling with kernel-sources. Also it makes it much easier to test these low-level work, so a community can spawn which informally shares low-level hacks, giving a much bigger momentum for low-level work.

And it allows for containment of potentially dangerous applications using subhurds. As a very simple example, I can open a webbrowser without giving it access to the internet and just add that capability later, when I really want to go online (as opposed to just showing local files).

But mainly: 

settrans -a ftp\: /hurd/hostmux /hurd/ftpfs /
dpkg -i ftp://ftp.gnu.org/…/*.deb

And that’s only the beginning.

Are you subscribed to the [email protected] mailing list? (See http://lists.gnu.org/mailman/listinfo/bug-hurd )

Yes :)

Do you have a permanent internet connection, especially during the time of the summer session? Are you able and willing to hang out on the Hurd IRC channel regularly? (As in: Running the IRC client more or less permanently and checking for activity now and then.) If it turns out that your mentor lives in a different time zone, could you shift your day/night rhythm to better match that of your mentor and other Hurd developers?

Yes, a permanent internet connection as well as a permanently running computer. Since I’m used to also work later in the evening (on hobby projects), the time zone should not be a major issue.

When does your university term end, when are your exams, and when does the next term begin?

I have a clean timetable for the summer: No exams anymore.

How much time do you intend to spend on your GSoC project per day/week during the summer months?

I plan to spend at least 40 hours per week on the PyHurd.

What other major activities will you engage in during the summer? (Moving apartments, longer vacations, other obligations, etc.) If any, how do you intend to make sure you will be able to dedicate sufficient time to your project nevertheless?

Finding a job for after the GSoC. This should not take too much time, all in all, but rather mean short out-times now and then.

How do you intend to make sure that your code will keep on being maintained and supported properly after the end of the GSoC program?

My main plan to keep it maintained is to comment it cleanly, and naturally to keep using the Hurd and PyHurd itself, so any breakage will bother me personally.

Also i want to get it merged into the main git repositories, so it is directly accessible for later developers.

Anything else you want to add to your application?

I’d love to work on PyHurd, because it grips me more and more. For example a high level API might get as simple as 

from translator.source.text import *
from translator.repres.tree import *
def source_text_changed(text): … (adapt tree object)
def repres_tree_changed(tree): … (adapt text object)
→ 2-way connectingk,5
writeonly is then done by simply leaving out the definition for the source_<whatever>_changed.
source is the node below and repres is the translated node