Posts for category 'Programming'

Jumbo

Today, I'm releasing something that I've wanted to release for a very long time. It's a project that I worked on during my Ph.D., and while I don't think it'll be terribly useful to anyone, a lot of work went into it that I want to preserve, even if just for myself.

That project is Jumbo, and it's now availabe on GitHub in two flavors: Jumbo for .Net 6+, and the original for .Net Framework and Mono. If you want to play around with it or learn more about it, you probably want the former.

Jumbo is an experimental large-scale distributed data processing system, inspired by MapReduce and in particular Hadoop 1.0. Jumbo was created as a way for me to learn about these systems, and should be treated as such. It's not production quality code, and you probably shouldn't entrust important data to it.

Basically, back when I was getting started with my Ph.D. in 2008, I found myself staring at the code of Hadoop (which wasn't even at version 1.0 yet at the time), and finding I wasn't really getting a good feel of how the whole thing fit together, and what really goes into designing a system like that.

So, some people at my lab suggested I should try building something for myself, which I did. I built, from the ground up, a distributed file system and data processing system, which is Jumbo. It was heavily inspired by Hadoop, and definitely borrows from its design (although no actual code was borrowed). In some aspects, I deviate from Hadoop quite a lot (especially since Jumbo isn't constrained to only using MapReduce).

Building Jumbo taught me a lot: about software design, about distributed processing, about decisions that affect scalability, and more. It's my hope that maybe, someone else interested in these topics might want to look at it and find what I did interesting. If nothing else, I just want to preserve this massive project that I did (still the biggest project I've done where I'm the sole contributor), and have its history available.

I did end up using Jumbo for some research efforts, which you can read about in a few papers as well as my dissertation under the University section of my site.

Jumbo is also the origin of one of my most widely used libraries, Ookii.CommandLine, so it's significant in that respect as well.

Like I said, I've wanted to release Jumbo for a long time. If you look through the original project's commit history you can see a bunch of work done in early 2013 (as I was nearing the end of my Ph.D.) like cleaning stuff up and adding documentation, but I never quite reached a level where I was comfortable doing so. The project, which primarily targeted Mono to run on Linux, wasn't that easy to set up and run.

In 2019, I ported the project to .Net Core, just to see if I could. That version was easier to play around with, and I wanted to release it then too, but I never quite got around to finishing it, until now.

So now, you can look at Jumbo and play around with it on .Net 6+, thanks to this new version. I've also expanded the documentation significantly, so it should be easy to get started and to learn more about how it works. The original Jumbo project for Mono and .Net Framework is only provided to preserve the original history of the project (the new repository only contains the history of the port). You probably shouldn't try and run it (though I obviously can't stop you).

If you want to comment on Jumbo or ask any questions, please use the discussions page on GitHub.

I will be writing a few more blog posts over the coming time about Jumbo and its history, which I will link to here as they become available.

Categories: University, Software, Programming
Posted on: 2022-09-20 23:54 UTC. Show comments (0)

Ookii.FormatC 2.3

After I recently released an updated version of Ookii.CommandLine, I figured Ookii.FormatC could also use some love.

This version comes with an optional new dark mode stylesheet, nullable reference types enabled for the library, the ability to write directly to a TextWriter, C# 10.0 keyword support, and a few minor other features and fixes.

Thanks to the ability to write to a TextWriter, you can now do stuff like this:

var formatter = new CodeFormatter()
{
    FormattingInfo = new CSharpFormattingInfo()
};

formatter.FormatCode(SampleCode, Console.Out);

Okay, writing HTML to the console is maybe not the most useful example, but you get the idea.

You can try it out with .Net Fiddle, or look at a sample that also shows the new dark mode in action. The online syntax highlighter has also been updated, and now supports PSParser based PowerShell formatting again.

And yeah, the NuGet package is version 2.3.1, rather than 2.3.0. That's because somehow the package for 2.3.0 ended up with an outdated binary in it. Not sure how that's possible, but it happened.

Categories: Software, Programming
Posted on: 2022-09-14 05:40 UTC. Show comments (0)

Ookii.CommandLine 2.4

I've released an update to Ookii.CommandLine, my library for parsing command line arguments for .Net.

This new version comes with nullable reference type support (for .Net 6+), a new helper to make parsing easier, more customizability, an easier way to make -Help style arguments, and some bug fixes.

See the full list of changes here.

With the new helper method, you can now just do the following to parse the arguments and write errors and usage to the console if parsing failed:

var parsed = CommandLineParser.Parse<MyArguments>(args);

And if you want to customize parsing behavior, you can still do so with this method:

var options = new ParseOptions()
{
    NameValueSeparator = '='
};

var parsed = CommandLineParser.Parse<MyArguments>(args, options);

Of course, existing code to parse arguments that manually creates an instance of CommandLineParser will continue to work.

Check it out on NuGet or GitHub, or try it out online!

Also, the Visual Studio code snippets (which previously required manual installation) are now available on the Visual Studio marketplace.

Categories: Software, Programming
Posted on: 2022-09-06 03:05 UTC. Show comments (0)

.Net Configuration Section Documentation Generator

Today I'm releasing another small tool that I created for personal use that I believe others might find of use. The tool is the .Net Configuration Section Documentation Generator (more proof that I should not be put in charge of naming things), which does what it says on the tin: it generates documentation for configuration sections defined using .Net's ConfigurationSection class.

Basically, it can generate an XSD schema from a ConfigurationSection, to which you can then add annotations to document the elements and attributes of the section, and then generate an HTML documentation file from that schema. The tool can be used with Microsoft .Net and Mono.

As an example, check out the schema for the documentation generator's own configuration section as generated by the documentation generator, and with added annotations. And finally, the documentation file generated from the annotated schema. This is only a very trivial configuration section, but it gives you an idea of what the output of the documentation generator looks like.

More information and downloads here.

Categories: Software, Programming
Posted on: 2013-06-08 10:42 UTC. Show comments (1)

Creating a route for a .asmx Web Service with ASP.NET routing

While working on the update to my site (which is now completely finished), I came across a minor problem: I had an old ASP.NET Web Service (you know, the .asmx type, predating even WCF) that I needed to preserve at the same URL for compatibility purposes.

However, that URL was inside a folder on the old site. And in the new ASP.NET MVC site, that folder didn't exist anymore; instead, requests for that path were handled by an MVC Controller. This meant that it was not possible to leave the .asmx file at its original location, because an existing physical directory pre-empts the MVC route for the controller, so it would render all the other pages with the same base URL unviewable.

I figured I'd be able to use RouteCollection.MapPageRoute to redirect the old URL to the .asmx file's new location, but it turned out that this function only works for pages (.aspx files), as it requires that the HTTP handler for the route's target derives from System.Web.Page. Searching the web wasn't able to find a solution for this, but some investigation into the inner workings of ASP.NET routing let me devise my own solution, which I thought I'd share here.

The trick, it turns out, is to create a custom IRouteHandler, one that is able to return the correct IHttpHandler for web services. This turned out to be fairly simple:

using System;
using System.Web;
using System.Web.Routing;
using System.Web.Services.Protocols;

public class ServiceRouteHandler : IRouteHandler
{
    private readonly string _virtualPath;
    private readonly WebServiceHandlerFactory _handlerFactory = new WebServiceHandlerFactory();

    public ServiceRouteHandler(string virtualPath)
    {
        if( virtualPath == null )
            throw new ArgumentNullException("virtualPath");
        if( !virtualPath.StartsWith("~/") )
            throw new ArgumentException("Virtual path must start with ~/", "virtualPath");
        _virtualPath = virtualPath;
    }

    public IHttpHandler GetHttpHandler(RequestContext requestContext)
    {
        // Note: can't pass requestContext.HttpContext as the first parameter because that's
        // type HttpContextBase, while GetHandler wants HttpContext.
        return _handlerFactory.GetHandler(HttpContext.Current, requestContext.HttpContext.Request.HttpMethod, _virtualPath, requestContext.HttpContext.Server.MapPath(_virtualPath));
    }
}

This route handler can then be used to add a route for a .asmx web service:

routes.Add("RouteName", new Route("path/to/your/service", new RouteValueDictionary() { { "controller", null }, { "action", null } }, new ServiceRouteHandler("~/actualservice.asmx")));

One important thing to note is the RouteValueDictionary values I'm passing to the route. If you don't add values for "controller" and "action" to the dictionary, the Html.ActionLink helper method will match this route for any controller or action, leading to completely incorrect URLs if this is the first matching route. Since you probably want to have this route before the MVC default route (which was the case for me), that would be a problem. Setting the values in the route value dictionary resolves this issue.

Of course, you can create your own extension method to make this easier:

public static Route MapServiceRoute(this RouteCollection routes, string routeName, string url, string virtualPath)
{
    if( routes == null )
        throw new ArgumentNullException("routes");
    Route route = new Route(url, new RouteValueDictionary() { { "controller", null }, { "action", null } }, new ServiceRouteHandler(virtualPath));
    routes.Add(routeName, route);
    return route;
}

Which means you can now simply use this:

routes.MapServiceRoute("RouteName", "path/to/your/service", "~/actualservice.asmx");

And voilà, I was now able to preserve the URL to my legacy web service without needing to have a physical file matching that URL. If you're faced with a similar situation, I hope this helps you.

Categories: Programming
Posted on: 2013-05-30 15:56 UTC. Show comments (4)

Latest posts

Categories

Archive

Syndication

RSS Subscribe

;