Ugly Perl: A lesson in the importance of API design

Posted Sunday, April 19th at 6:33p.m. under Perl


One of the most challenging aspects of my role as a Software Architect has been trying to standardise our development practices in a language as flexible as Perl. Powered by the culture of There is More Than One Way to Do It and a previous lack of technical leadership, we have ended up in a situation where almost every conceivable style of code has been utilised across all of our different mod_perl web applications. It's a maintenance nightmare.

Last year I started to turn that around by advocating object oriented programming and model view controller architecture using modern Perl techniques powered by Moose. Something I noticed though when we sat down and evaluated the state of our code was that some of the easiest and most reliable systems we had were the procedural ones, and by far the most painful systems to work with on a daily basis were those where the lead developer had went on an architectural expedition and built a tightly-coupled OOP monstrosity.

What we took away from that evaluation process is that, more than choice of programming language, more than choice of development paradigm and more than choice of development methodology, good API design is the single most important factor in how developers perceive the quality of code. Introducing the code review process to our team only confirmed this: almost every review I've taken part in has revolved around naming conventions and style rather than performance or implementation.

And it's no coincidence that by focussing less on concepts like OOP and MVC and more on creating good APIs, we've really started to turn the corner in the land of maintenance programming.

API in action: jQuery

One of the best examples of the impact a good API on a language is in the Javascript framework community. When jQuery launched, it was a textbook case of reinventing the wheel. It was slower than most of the competitors, had less features and there was no community that compared with the following that YUI and Prototype had built up. Yet somehow it still managed to muscle its way into becoming synonymous with web 2.0.

$('#my-link').click(function(e) { 
    $('#content').addClass('active').slideUp();
});

The secret to its success was in the only area where it introduced something new: the slick and concise CSS selector API. No new functionality, no performance benefits, the selling point from day one of jQuery has been "designed to change the way that you write Javascript." And people really bought into it.

jQuery really is a great example of how a solid framework API can turn around the whole perception of working with a language. Three years ago Javascript had a nasty reputation of being frustrating and difficult to work with, these days whether it's jQuery or Dojo or MooTools, the elegant APIs of the most popular frameworks more often than not make it a pleasure.

The Perl ORM problem

The motivation for this post came when I started to look for a Perl ORM solution to adopt for our MVC development. Moving away from Perl would have been too extreme and unrealistic a step, as we were going for slow and gradual refactoring of each system. So when I looked at the two most frequently discussed Perl solutions, DBIx::Class and Rose::DB::Object, what I saw was more than a little disappointing.

As a team we quickly came to the conclusion that it might be best to roll our own. Most of us had used the likes of ActiveRecord, the Django ORM and SQL Alchemy from the Ruby and Python communities, so we definitely felt like the syntax of an ORM could be a lot better than what we were seeing. When I mentioned in #moose a few months back that I was thinking of writing my own ORM, I was mocked quite heavily and pointed in the direction of Fey::ORM. Again, looking at the code all I could think of was that the standard was nowhere near what I had seen from other solutions.

In particular, I really liked using the Django ORM. I used Django to build this blog, as well as a few other personal projects and the syntax seemed to gracefully scale and handle almost all of the nasty edge cases. Trying to describe how it felt moving from the Django ORM to DBIx::Class is best illustrated with an example. The following models are directly from the DBIx::Class tutorial on CPAN.

package MyDatabase::Main;
use base qw/DBIx::Class::Schema/;
__PACKAGE__->load_namespaces;

1;

package MyDatabase::Main::Result::Artist;
use base qw/DBIx::Class/;
__PACKAGE__->load_components(qw/PK::Auto Core/);
__PACKAGE__->table('artist');
__PACKAGE__->add_columns(qw/ artistid name /);
__PACKAGE__->set_primary_key('artistid');
__PACKAGE__->has_many('cds' => 'MyDatabase::Main::Result::Cd');
1;

package MyDatabase::Main::Result::Cd;
use base qw/DBIx::Class/;
__PACKAGE__->load_components(qw/PK::Auto Core/);
__PACKAGE__->table('cd');
__PACKAGE__->add_columns(qw/ cdid artist title/);
__PACKAGE__->set_primary_key('cdid');
__PACKAGE__->belongs_to('artist' => 'MyDatabase::Main::Result::Artist');
__PACKAGE__->has_many('tracks' => 'MyDatabase::Main::Result::Track');
1;

package MyDatabase::Main::Result::Track;
use base qw/DBIx::Class/;
__PACKAGE__->load_components(qw/PK::Auto Core/);
__PACKAGE__->table('track');
__PACKAGE__->add_columns(qw/ trackid cd title/);
__PACKAGE__->set_primary_key('trackid');
__PACKAGE__->belongs_to('cd' => 'MyDatabase::Main::Result::Cd');
1;

The repetition of the Perl special variable __PACKAGE__ immediately stands out, but the rest isn't too bad. For me though, the django equivalent is far superior:

from django.db import models

class Artist(models.Model):
  artistid = models.AutoField(primary_key = True)
  name = models.CharField()

class Cd(models.Model):
  cdid   = models.AutoField(primary_key = True)
  title  = models.CharField()
  year   = models.YearField()
  artist = models.ForeignKey(Artist)

class Track(models.Model):
  trackid = models.AutoField(primary_key = True)
  title   = models.CharField()
  cd      = models.ForeignKey(Cd)

The first thing that stands out about the django syntax is that it very similar to what it represents: the definition of a SQL table. In fact, the nature of the django model definition is that you provide all the information required to actually run a CREATE TABLE command if you need to. In one lean Python class we have everything we need to know about our model.

A notable difference between the two packages is in the way they handle the bidirectional ForeignKey relationship: you only have to define them once in Django, whereas in DBIx::Class you need to define both directions separately. You could argue that in DBIx::Class this is intentional for readability – when you load up each model you see all of that model's relationships inside its own package – but in real, complex systems this would actually lead to major model definition bloat.

When it comes to querying these models, the design gap is even more apparent. The DBIx::Class tutorial has this as example CRUD code:

  my $schema = MyDatabase::Main->connect('dbi:SQLite:db/example.db');

  my @artists = (['Michael Jackson'], ['Eminem']);
  $schema->populate('Artist', [
     [qw/name/],
     @artists,
  ]);

  my %albums = (
    'Thriller' => 'Michael Jackson',
    'Bad' => 'Michael Jackson',
    'The Marshall Mathers LP' => 'Eminem',
  );

  my @cds;
  foreach my $lp (keys %albums) {
    my $artist = $schema->resultset('Artist')->search({
      name => $albums{$lp}
    });
    push @cds, [$lp, $artist->first];
  }

  $schema->populate('Cd', [
    [qw/title artist/],
    @cds,
  ]);


  my %tracks = (
    'Beat It'         => 'Thriller',
    'Billie Jean'     => 'Thriller',
    'Dirty Diana'     => 'Bad',
    'Smooth Criminal' => 'Bad',
    'Leave Me Alone'  => 'Bad',
    'Stan'            => 'The Marshall Mathers LP',
    'The Way I Am'    => 'The Marshall Mathers LP',
  );

  my @tracks;
  foreach my $track (keys %tracks) {
    my $cdname = $schema->resultset('Cd')->search({
      title => $tracks{$track},
    });
    push @tracks, [$cdname->first, $track];
  }

  $schema->populate('Track',[
    [qw/cd title/],
    @tracks,
  ]);

Even though I know what they're trying to achieve here, looking at the code it still confuses me as to why they need to search a result set to connect these models together. Once again, when compared to the equivalent django API, the difference speaks for itself:

  from myapp.models import Artists, Albums, Tracks;
 
  mj = Artists.create(name = 'Michael Jackson);
  eminem = Artists.create(name = 'Eminem')

  thriller = Albums.create(title = 'Thriller', artist = mj)
  bad = Albums.create(title = 'Bad', artist = mj)
  themmlp = Albums.create(title = 'The Marshall Mathers LP', artist = eminem)
  Tracks.create(title = 'Beat It', album = thriller)
  Tracks.create(title = 'Billie Jean', album = thriller)
  Tracks.create(title = 'Dirty Diana', album = bad)
  Tracks.create(title = 'Smooth Criminal', album = bad)
  Tracks.create(title = 'Leave Me Alone', album = bad)
  Tracks.create(title = 'Stan', album = themmlp)
  Tracks.create(title = 'The Way I Am', album = themmlp)

Simple, clean and self-documenting.

Singling out DBIx::Class might also seem a little snide, but I really don't like sitting here criticising open source code. DBIx::Class solves a difficult problem and the authors have contributed far more to the Perl and open source communities than I have. The library is also over four years old and has had to maintain backwards compatability throughout that time. The original designer might have solved the ORM problem differently today.

Still, the state of affairs is that the elegant and evolving solutions in other languages are exactly what the Perl community wants to compete with. When I read posts like this one claiming that Catalyst and DBIx::Class are a demonstration of what Perl can do in 2009 I can only think that a lot of these guys are missing the point. Competing against feature sets just isn't enough anymore: frameworks like jQuery, Rails and Django have set the expectation - developers want clean APIs. And the frustrating thing is that Perl is the perfect language to deliver them.

Modern Perl Case Study: Sugar ORM

To illustrate the point (and so those whose code I've criticised can tear me a new one back!), I'll show you what my own solution to the ORM problem has been. Let's reuse the same tables as before and create a new interface with the help of Moose sugar.

Model definitions:

package models::base;
use base 'ORM::Table';

__PACKAGE__->add_connection(username => 'david', password => 'thdnsn484');

package models::artists;
use Sugar::ORM;
extends 'models::base';

field 'artistid' => (type => 'Int', primary_key => 1);
field 'name'     => (type => 'Int');
1;

package models::cds;
use Sugar::ORM;
extends 'models::base';

field 'cdid'  => (type => 'Int', primary_key => 1);
field 'title' => (type => 'Char');
foreign_key 'artist' => (through => 'models::artist', reverse => 'albums');
1;

package models::tracks;
use Sugar::ORM;
extends 'models::base';

field 'trackid' => (type => 'Int', primary_key => 1);
field 'title'   => (type => 'Char');
foreign_key 'album' => (through => 'models::cd');
1;

CRUD operations:

use aliased 'models::artists' => 'Artist';
use aliased 'models::cds' => 'Album';
use aliased 'models::tracks' => 'Song';

my $mj = Artist->create(name => 'Michael Jackson');
my $eminem = Artist->create(name => 'Eminem');

my $thriller = Album->create(title => 'Thriller', artist => $mj);
my $bad = Album->create(title => 'Bad', artist => $mj);
my $themmlp = Album->create(title => 'The Marshall Mathers LP', artist => $eminem);

Song->create(title =>'Beat It', album => $thriller);
Song->create(title => 'Billie Jean', album => $thriller);
Song->create(title => 'Dirty Diana', album => $bad);
Song->create(title => 'Smooth Criminal', album => $bad);
Song->create(title => 'Leave Me Alone', album => $bad);
Song->create(title => 'Stan', album => $themmlp);
Song->create(title => 'The Way I Am', album => $themmlp);

And, as a bonus some extra code to traverse the relationships:

for my $artist ($mj, $eminem)
{
    say $artist->name;
    for my $album ($artist->albums)
    {
        say $album->title;
        say join ',', $album->tracks;
    }
}

Design Explanation

One of the reasons the Django model definition is so much trimmer than DBIx::Class is because Python supports shared class data natively and Perl doesn't. This is important for our models' performance because you only want to load up the fields and relationships once at compile time rather than at each object instantiation. To compensate, Sugar::ORM models are just Moose classes that use Moose::Exporter to add table, field, foreign_key and many_to_many to the package namespace. Each one of those subroutines in turn stores information about the model as metadata at compile time.

One key difference from django is that you have to define a base class to provide database connectivity, this could just as easily be refactored using roles. This decision was made because of how tightly coupled to the Django web environment the ORM is. If you want to access your models outside of a Django web request, like in a cron script, then you have a bit of leg work to do to set up the django environment. To avoid this and decouple the ORM entirely from the web, all configuration for the ORM talking to the database is completely self-contained.

Since the sample models here are pretty basic, here is a more complex one to illustrate how Sugar::ORM handles some common edge cases (of package names not matching the table name or foreign key field names being different from the primary key of the table they relate to, etc.):

package blog::models::posts;
use Sugar::ORM;
extends 'model::base';

table 'blogposts';
field 'id'               => (type => 'Int', primary_key => 1);
field 'title'            => (type => 'Char', max_length => 128, required => 1);
field 'slug'             => (type => 'Slug', from => 'title');
field 'datetime_posted'  => (type => 'DateTime', auto_on_add => 1);
field 'datetime_updated' => (type => 'DateTime', auto_now => 1);
field 'post'             => (type => 'Text', required => 1);
foreign_key 'author'     => (from => 'added_by_id', through => 'blog::models::user');
many_to_many 'tags'      => (through => 'blog::models::tags', using => 'blogposts_tags');
has 'something_extra'    => (is  => 'rw', isa => 'Bool', default => sub { 0; });

sub permalink
{
    my $self = shift;
    return sprint("/post/%s", $self->slug);
}

The equivalent model in Django would be:

class Posts(models.Model):
    title            = models.CharField(max_length = 128, required = 1)
    slug             = models.SlugField(max_length = 128)
    datetime_posted  = models.DateTimeField(auto_on_add = 1)
    datetime_updated = models.DateTimeField(auto_now = 1)
    post             = models.TextField(required = 1)
    added_by         = models.ForeignKey(User, rel_name = 'author')
    tags             = models.ManyToManyField(Tag)
    something_extra  = None

    def permalink(self):
        return "/post/%s" % self.slug
    class Meta:
        db_table = 'blogposts'

When you compare the two, there is very little difference in terms of readability except perhaps the symbol bloat in the Perl version. But what is interesting is that the DBIx::Class version is actually much trimmer because of how little information is stored about columns:

package MyDatabase::Main::Result::Posts;
use base qw/DBIx::Class/;
__PACKAGE__->load_components(qw/PK::Auto Core/);
__PACKAGE__->table('blogposts');
__PACKAGE__->add_columns(qw/ id title slug datetime_posted datetime_updated post /);
__PACKAGE__->set_primary_key('id');
__PACKAGE__->belongs_to('author' => 'MyDatabase::Main::Result::User');
__PACKAGE__->many_to_many('tags' => 'blogposts_tags', 'post');

This is a good example of why conciseness isn't everything.  With Django and Sugar::ORM, everything you'd need to know about the SQL table is stored in the model – that way to understand the models all you need to do is read the code. With DBIx::Class I suspect there would be a whole load of DESC `table` commands at the RDBMS command line.  Plus, by storing the field types in the model code you can also do useful things later, like add data integrity checks during CRUD operations.

Of course neither the django ORM nor Sugar::ORM are perfect, and with something non-deterministic like this there will always be some people who prefer the DBIx code. Putting that to one side, what I've tried to demonstrate with my API design is that Perl is incredibly flexible. If it has a reputation as being ugly, then all that says is that the Perl community has been writing ugly APIs.

 

Enter the beauty contest

The good news for Perl is that one of the major causes of all the bad code out there, its flexibility, also makes it one of the most powerful languages around for creating amazing APIs with. Armed with Moose and tools like Moose::Exporter and MooseX::Declare, there is no limit to the elegant and clean Perl code that we could all be writing.

The community definitely has the talent, which is why it would be such a shame if Perl lost out in the beauty contest. But if the Perl community is really serious about a modern Perl rennaisance, then there is a very real need to get serious about API design.

 

 

Update:

Jess Robinson pointed out that the DBIx::Class tutorial I used was not the most elegant way to use the API. Her updated code adds field types to the model definition as well as provides a much cleaner insertion API:

package MyDatabase::Main::Result::Artist;
use base qw/DBIx::Class/;
__PACKAGE__->load_components(qw/PK::Auto Core/);
__PACKAGE__->table('artist');
__PACKAGE__ ->add_columns(
artistid => {
data_type => 'integer',
is_auto_increment => 1,
},
name => {
data_type => 'varchar',
size => 255,
}
);
__PACKAGE__->set_primary_key('artistid');
__PACKAGE__->has_many('cds' => 'MyDatabase::Main::Result::Cd');
1;

 

my $schema = MyDatabase::Main->connect('dbi:SQLite:db/example.db');

my $mj = $schema->create({ name => 'Michael Jackson' });
my $eminem = $schema->resultset('Artist')->create({ name => 'Eninem' });

my $thriller = $schema->resultset('Album')->create({
artist => $mj,
title => 'Thriller',
});

my $bad = $schema->resultset('Album')->create({
artist => $mj,
title => 'Bad',
});

my $themmlp = $schema->resultset('Album')->create({
artist => $eninem,
title => 'The Marshall Mathers LP',
});

$schema->resultset('Track')->create({ title => 'Beat Id', album => $thriller })

Clearly the original comparison I made was a case of apples and oranges as populate in the DBIx::Class API is actually a multi-insert interface which django doesn't provide. It's also clear from this example now that resultset selects the name of the model to work with. Perhaps this illustrates that good documentation is just as important as a sexy API.

Posted by David McLaughlin on Sunday, April 19th

25 Responses


1

Aaron Trevena

Monday, April 20th

You're writing your own ORM - that's so cute ;)

Anyway - I think you're worrying way to much about what people write and maintain widely used frameworks refer to as "scaffolding" - basically the training wheels that beginners use and then remove once they've got the hang of the system.

I'd be a lot more concerned about how the ORM works in the last 90% of your project than the first 30 minutes.

2

David McLaughlin

Monday, April 20th

Well as I explained, we're writing our own because no one on our team wanted to use existing solutions.

'Perl is dead' is like a programming news aggregator meme at this point, people keep asking the question why Perl is struggling, so I'm using the fact that we have to roll our own ORM to answer it: most of the APIs are horrible.

I'm not sure you're quite in touch with how django is used either. Those models aren't temporary scaffolding at all, they are designed to provide access to almost everything you'd want to do with your data.

There is of course people who do a lot of business logic on the database side or perform really advanced SQL. These are the same people who choose SQL Alchemy over the Django ORM.

I'm designing this ORM so that it can replace 90% of our queries. The other 10% of queries that we're happy to fall back to raw SQL for are very rarely used in actual applications, instead being used in reporting tools or for summary table maintenance.

It's not that big of a job either, since we're only targetting one database system at this point. We use MySQL and the other quirk is the fact that MySQL easily handles queries that span multiple databases - we use this a lot.

The system handles our 90%, but thanks for the concern.

3

Tobias Hoellrich

Monday, April 20th

Nice - I like it. As a newcomer to Catalyst, I'm also struggling with the complex syntax of the ORMs supported by it. DBIx::Class seems to be the most widely accepted solution at the moment, but it sure ain't elegant.

When did you say you were going to upload Sugar::ORM to CPAN? ;-)

4

David McLaughlin

Monday, April 20th

Tobias:

I know you're probably just kidding about the CPAN thing, but I'll answer your question anyway:

There's two major obstacles to open sourcing the code.

The first is that I've wrote this code on company time. I'll definitely need to speak to management to see if they support releasing this code as open source. It's not a discussion I've had yet as this mostly been a 20% project thus far.

The second reason is that the only reason I was crazy enough to try and write my own ORM is that when you're only targetting one database engine it's not TOO big of a job. I could release the code and say "MySQL only until someone wants to support other databases" but then I'd also want to design the interface to support multiple engines first.

But it's something I definitely hope to do once we test it hard internally.

5

Jess Robinson

Monday, April 20th

Hi David! Thanks for pointing out these inadequacies in our docs.. especially Manual::Example, which I feel like burning now ;)

Let me clear up a few wrong impressions you seem to have gotten from there:

1) Myth: DBIx::Class doesn't define all the data you'd need to create SQL tables.
- Yes it does. The ->add_columns (qw/artistid name/) in the Example is just a short form. The long form is (see DBIx::Class::ResultSource docs):

  ->add_columns(
artistid => {
data_type => 'integer',
is_auto_increment => 1,
},
name => {
data_type => 'varchar',
size => 255,
}
);

Which as you can see provides all the bits to create SQL CREATE TABLE statements. And in fact if you have SQL::Translator installed, you can call $schema->deploy() and have it create the tables for you. (Or $schema->create_ddl_dir() and get the SQL for it dumped to file if you prefer). For bonus points, create_ddl_dir will diff with a previous dump if you ask it to, and produce the ALTER TABLE statements.

2) Need to do a search to create new rows.
- I can see where the example gives that impression. What its actually doing is inserting the artists, then refetching them in order to grab the database-auto-incremented id field, there are better ways to do this.

Jess - chief DBIx::Class doc wrangler, off to kill ::Example for giving us a bad rep now ;)

6

Jess Robinson

Monday, April 20th

Hmm.. I hit preview, and it posted. Imagine theres saneish indented formatting in my last comment!

Jess

7

David McLaughlin

Monday, April 20th

Hi Jess,

Thanks for the post.

The code in your comment is a massive improvement over the tutorial in the CPAN distribution. I guess my confusion stems from the naming conventions - "resultset" being used to specify which class to work with. It's amazing how your updated code makes it that much clearer.

Apologies for the crappy comments system too... I hadn't actually planned on anyone using it! But I'll move your improved code directly into my post.

David

8

Jess Robinson

Monday, April 20th

Thanks David!

Hmm, you lost the best bit:


my $mj = $schema->resultset('Artist')->create({
name => 'Michael Jackson',
albums => [ {
title => 'Thriller',
tracks => [
{ title => 'Beat It', }
{ title => 'Billie Jean', }
],
}, {
title => 'Bad',
tracks => [
{ title => 'Dirty Diana' },
{ title => 'Smooth Criminal' },
{ title => 'Leave Me Alone' },
],
} ]
});

Also, minor hint: "She" ;)

Jess

9

Matt S Trout

Monday, April 20th

The reason for the resultset name is that you aren't asking to work with a class - you're asking to work with a resultset of all objects of that class persisted in the current database.

This distinction doesn't matter until you need to connect to two databases with similar schemas within one application. Then it does.

It's also why your proposed API's class-based-ness is a really, *really* bad idea - we learned this lesson from Class::DBI, it's a shame people seem to keep making the same mistake.

I think you should still show your new sketch in the post, it was kind of pretty - and I'm glad you like MooseX::Declare, I'm somewhat involved in that as well as in DBIC :)

I'd also comment that I still find your new code a bad idea, because it still makes the same mistake that DBIC did - including lots of table metadata in the classes. What we're planning to do with DBIx::Class 0.09 is to allow plain Moose classes to be persisted by defining a mapping elsewhere (or in the class, or by letting DBIC do it automatically for you). Perhaps rather than wasting your internal manpower re-inventing the wheel you could help us build a round one on CPAN - we'd be delighted to see you and your colleagues on irc.perl.org #dbix-class and the dbix-class mailing list. I'll comment again later with the URL to my slides from the Nordic Perl Workshop a few days ago talking about the future of DBIx::Class and what we have in mind.

10

Matt S Trout

Monday, April 20th

And, of course, I'm an idiot and you did leave your code in. But let me show you what I was thinking about, roughly, for new APIs -


class Music {
has artists => (is => 'ro', isa => Set[Artist]);
}

class Artist {
has name => (is => 'rw', isa => 'Str');
has cds => (is => 'ro', isa => Set[CD]);
}

class CD {
has name => (is => 'rw', isa => 'Str');
}

my $music = Data::Store->new(dsn => $dsn)->create_persisted('Music');

my $mj = $music->artists->add({ name => 'Michael Jackson' });

$mj->cds->add({ name => $_ }) for qw(Thriller Bad);

(if the pre tags were the wrong thing, please do edit my post :)

11

David McLaughlin

Monday, April 20th

@Jess

Oops, fixed the gender typo. Apologies :)

That code that you posted is great for situations when it's all new data coming through, but I have to admit I'm starting to remember why I was put off DBIx::Class in the first instance: the number of different ways to do the same thing.

Flexibility for most people is obviously a major plus, but I guess we're in the 'consolidation' style of mind at the moment. The pain of dealing with TIMTOWTDI legacy code is still ongoing, so perhaps that's why so many of us in here are appealed by frameworks which attempt to impose a philosophy on you.

@ Matt

It's pretty clear to me at this point that bringing up the DBIx::Class interface was a mistake. I should really have looked into it more before passing any sort of judgement.

I knew it when I posted, but if I had any doubts as to why the DBIx::Class interface is the way it is, it's because DBIx::Class is designed to solve 90% of general database problems. Obviously my ORM solution is designed to solve 90% of the problems a small development team encounters in their small bubble on the internet, so it's no surprise that I've been able to come up with a more elegant API.

I'd be real interested in discussing this more with you though. I'll try and pop into IRC at some point (after I've taken a much closer look at DBIx::Class).

12

Matt S Trout

Monday, April 20th

Assuming we manage to make the sketch I showed work, I'd argue that DBIC 0.09 is going to be substantially more elegant than the code that you showed - the only thing I'm really requiring is that you have an object rather than class methods, which is merely good design - and I have some interesting ideas about managing to provide class-method-ish stuff anyway - something like


use MusicDB export => [':all'], dsn => $dsn;

Artists->add(...);

by exporting a


sub Artists { $music->artists }

and then making $music a global somewhere so it can be local()ed, which will enable


with_connection(
dsn => $other_dsn,
sub { ... }
);

if you find yourself needing a second connection later.

We're also hopefully in the process going to consolidate things so that instead of having lots of ways to do things, there's one sufficiently flexible way.

I do appreciate that what you currently have is better than what we currently have, but that's the nature of working on a large problem space - it takes a while to get a complete new solution. On the upside, I shipped 0.08100 two days ago so the 08 series can now go back into maintenance mode and active development shift to 09, so please turn up soon so you can provide input before the design is finalized.

13

John

Monday, April 20th

An API that is verbose and explicit can always be wrapped in a simpler one that makes more assumptions, but the reverse is not true. That said, even the APIs that you found distasteful can look a lot like your Python examples.

For example, here's a possible Rose::DB::Object equivalent of your classes:

 
package Artist;
use base 'Rose::DB::Object';

__PACKAGE__->meta->setup(
columns => [
id => { type => 'serial' },
name => { type => 'varchar' },
]
);

package Track;
use base 'Rose::DB::Object';

__PACKAGE__->meta->setup(
columns => [
id => { type => 'serial' },
title => { type => 'varchar' },
album_id => { type => 'int' },
],
foreign_keys => [ 'album' ],
);

package Album;
use base 'Rose::DB::Object';

__PACKAGE__->meta->setup(
columns => [
id => { type => 'serial' },
title => { type => 'varchar' },
year => { type => 'int' },
artist_id => { type => 'int' },
],
foreign_keys => [ 'artist' ],
relationships => [ 'tracks' => 'one to many' ],
);

Of course, if we're golfing, an ActiveRecord-like introspection-based alternative also exists:


package Artist;
use base 'Rose::DB::Object';
__PACKAGE__->meta->setup(auto => 1);

package Track;
use base 'Rose::DB::Object';
__PACKAGE__->meta->setup(auto => 1);

package Album;
use base 'Rose::DB::Object';
__PACKAGE__->meta->setup(auto => 1);

14

John

Monday, April 20th

And here's your data insertion example:


$mj = Artist->new(name => 'Michael Jackson');
$eminem = Artist->new(name => 'Eminem');

$thriller = Album->new(title => 'Thriller', artist => $mj);
$bad = Album->new(title => 'Bad', artist => $mj);
$themmlp = Album->new(title => 'The Marshall Mathers LP', artist => $eminem);

$thriller->add_tracks({ title => 'Beat It' },
{ title => 'Billie Jean' });

$bad->add_tracks({ title => 'Dirty Diana' },
{ title => 'Smooth Criminal' },
{ title => 'Leave Me Alone' });

$themmlp->add_tracks({ title => 'Stan' },
{ title => 'The Way I Am' });

$thriller->save;
$bad->save;
$themmlp->save;


Although I would probably do it more like this:

$mj = Artist->new(name => 'Michael Jackson');
$eminem = Artist->new(name => 'Eminem');

$thriller = Album->new(
title => 'Thriller',
artist => $mj,
tracks => [
{ title => 'Beat It' },
{ title => 'Billie Jean' },
],
)->save;

$bad = Album->new(
title => 'Bad',
artist => $mj,
tracks => [
{ title => 'Dirty Diana' },
{ title => 'Smooth Criminal' },
{ title => 'Leave Me Alone' }
],
)->save;

$themmlp = Album->new(
title => 'The Marshall Mathers LP',
artist => $eminem,
tracks => [
{ title => 'Stan' },
{ title => 'The Way I Am' },
],
)->save;

15

John

Monday, April 20th

I tried to use the "Preview" button twice, and both times it posted immediately. I also looked for an email address for the author on this site and couldn't find one. (Is it staring me in the face?)

16

David McLaughlin

Monday, April 20th

@John

I need to sort out the comments system.. was kind of tacked on as an after-thought. People who think Perl are dead are nuts. I've made two posts on the subject of Perl and had almost 50 comments.

I guess need to add a contact page too. If you're still looking to contact me, then my email address is david [at] dmclaughlin.com.

17

kd

Tuesday, April 21st

As the original author of the DBIx::Class::Manual::Example doc that worked (based on an incomplete example previously) I apologise for the error.

I don't know whether other ORMs can do this easily, but I found that the following script introspected on a 50 table sqlite database which I had no control over, successfully found all the relationships I needed, and allowed me to create a module which facilitated this database talking to another much simpler database which I needed to keep independent due to business logic constraints.


#!/usr/bin/perl
use warnings;
use strict;
use DBIx::Class::Schema::Loader qw/ make_schema_at /;

make_schema_at("Zotero::Schema",
{
# components => ['InflateColumn::DateTime'],
debug => 1,
relationships => 1,
dump_directory => './lib' ,
},
["dbi:SQLite:dbname=../zotero.sqlite", "",""]);

18

Dave Rolsky

Wednesday, April 22nd

You mention it, but did you actually look at Fey::ORM at all? It looks a _lot_ like your Sugar::ORM.

No, it doesn't let you define one has_X and generate methods in two classes, but that's some horrible action-at-a-distance. Of course, if you really wanted it, adding it would be fairly simple.

And as was pointed out by others, with DBIC you can have it figure out your schema definitions by introspecting the schema (much like Fey::ORM).

19

David McLaughlin

Thursday, April 23rd

@ Dave

One of the major reasons why Fey::ORM wasn't considered was this warning:


This is still very new software, and APIs may change in future releases without notice. You have been warned.

Moreover, this software is still missing a number of features which will be needed to make it more easily usable and competitive with other ORM packages.

I'm not looking for an ORM for a personal project, our sites are currently in production.

I also don't see too many similarities between Sugar::ORM and Fey::ORM, if there any similarities in how the code looks then you could also say that jQuery looks a LOT like Prototype. The difference in practice is huge.

20

Matt Kraai

Thursday, April 23rd

In the model definition for Sugar::ORM, shouldn't the name field not be a primary key?

21

David McLaughlin

Thursday, April 23rd

@Matt

Thanks, you're right, that's a typo.

22

Robert K

Saturday, April 25th

Interesting blog post thanks.

Still, writing your own ORM? Please don't do that. You'll be hating yourself because it'll be 8 at night on a Thursday and you'll be stuck in the office tracking down a bug of your own creation rather than out having a beer.

Personally, if I were in your situation, I'd much rather write a sugary API on top of DBIC than try to roll my own.

23

David McLaughlin

Sunday, April 26th

@Robert

I'm still not completely sure about how to progress. I have been leaning towards creating sugar over one of the existing solutions, so right now I'm just trying to implement a database model in each of those packages so that I can get a handle on their strengths and weaknesses. Then we'll take the final decision as a team.

24

Adam Kennedy

Tuesday, April 28th

Great post Dave.

I'm certainly no DBIx::Class lover, I've fought with Matt a few times over it.

I will say though that it's skill-set is most definitely biased towards the high end of the implementation bell curve.

A lot of the features in it are designed for dealing with large database with lots of tables, complex joins and so on.

The smaller and simpler your database, the easier it is to find sugar.

At the other extreme from DBIx::Class is my ORM ORLite, which only uses SQLite.

Here's all the code to set up your ORM classes with ORLite. One line of code triggers a schema inspection process which just inflates the entire API.

package Foo;

use ORLite 'my/sqlite.db';

And one set up, you get all the basic stuff.

my @adams = Foo::Person->select(
'where first_name = ?',
'Adam',
);

my $weirdo = Foo::Person->create(
first_name => 'Adam',
last_name => 'Kennedy',
);

25

Robert K

Tuesday, May 12th

To show you what can done, here is something I've whipped up since I saw your post:

http://github.com/robertkrimen/DBICx-Sucrose/tree/master


package t::Schema;

use DBICx::Sucrose;

table( 'Artist' => sub {

column 'name' => Text, NotNull;
column 'age' => Integer, Null;

} );

table( 'Cd' => sub {

column 'title' => Text, NotNull;
column 'SKU' => Type('CustomSKUType'), NotNull;

} );

The above will create a bonafide DBIx::Class::Schema that you can use as-usual