Friday, January 30, 2009

Maven Maven Maven

My post Maven Continues to Suck drew a number of comments, including a helpful and thoughtful comment from Jason van Zyl, who concluded "I don't think it's so much that Maven continues to suck as much as we need to do more than the free book we already have to get people past these simple setup problems that cause frustration."

I responded briefly in the comments but wanted to expand a bit.

Perhaps I will eventually see the light - I do keep hoping, because there doesn't seem much alternative - but I have not yet. Respectfully, Maven folks, I think it's optimistic to hope that if you "get people past these simple setup problems that cause frustration" you'll be in the clear. I've been working with Maven now for half a year; it continues to be perhaps my primary source of frustration, and to interfere in almost everything I do. I cannot say the same for any of the other tools I use. Neither my IDE (Eclipse), my version control system (SVN), my language (Java) nor its libraries were this problematic this far along.

The difference may in part be in my expectations; but the issues go beyond simple setup problems. To pick two other examples: (1) The Eclipse/Maven integration is rough, causing what should be simple Eclipse operations (saving files, jumping to referenced types, debugging) to be slower and less accurate. (2) No one on our team has yet figured a way to pipe arbitrary system properties from a Maven command line into a forked app server process in the context of a system test. These may be blamed on third-party software (M2Eclipse, Surefire); or on lack of Maven configuration skillz; but the Maven ecosystem is part of working with Maven, as is the fact that most developers on a team should not be expected to be Maven experts.

It is worth mentioning that invoking "mvn help" at the command line first tries to download stuff from teh internets, and then spits out a cryptic build failure error; mvn --help spits out a command line usage message that says nothing about how to actually get help. Even "mvn clean" tries to first do an update (which is always the wrong thing to do before doing a clean, because you may lose information about what to clean). Have you ever tried to use Maven without an internet connection handy, like on an airplane? Epic fail. I know about mvn -o but have never been able to get it to work, perhaps because the web of dependencies is so fragile and unstable.

By contrast, in our core code base we have a homebrew build script that solves all these problems, is easy for anyone with basic programming knowledge to maintain and modify, and just never seems to get in the way. If Maven requires more domain-specific knowledge, skill, and time to maintain than a hard-coded build script (or Ant script) would, is it buying us anything?

No tool is the right answer for all problems, and most tools are the right answer for some. Almost any tool can be extended, with sufficient skill and time, to do anything. That doesn't make it the right tool.

Moreover, the more smarts that we build into our Maven configuration, the more that we will rely on needing to hire developers with serious Maven chops; I would rather hire developers with serious programming chops. Maven skills do not generalize to other problems; Java, Groovy, Ruby, Perl do.

What I'm saying here is that I think there is a problem in principle with basing a build on a tool that requires deep domain-specific knowledge to use well; that I think Maven is such a tool; that, further, I think even with solid knowledge Maven is based on premises (such as the idea of a SNAPSHOT) that don't model the world well (pre-release output of a CI process is not equivalent to locally-built output of a local change); and that, finally, there is only room in the world for at most one convention-based tool, and we already have more than one.

Put differently, I'm saying that I think even if I knew how to use it well, Maven would not be the right tool.

18 comments:

Tim said...

I feel similarly about Maven, and 'Convention over Configuration' tools like Maven and Ruby on Rails in general. Their obvious advantage is that you need to write less redundant code/configuration. But this also requires you to know the tool very well, resulting in a steep learning curve.

It's usually worst in a team, when you have to deal with other people's code who may have used features that you don't know about. The code may be very concise, but the tool does so many things 'magically' in the background that it becomes very difficult to understand it. You can't simply go through the code step-by-step to find out what it does. You also need to know the conventions very well.

A major part of the problem is documentation. If you call a Java method or an Ant task, you can easily look up what happens in the API/task documentation. But if the tool does something by itself, without being specifically instructed to do so, how can you find out what caused this? If suddenly the value of a field changes and you neither called a method nor have any configuration file or Java annotation, where can you find an explanation in the documentation? There's certainly a reason for the change, maybe the name of your field matched the name of a database column, but this is something that's not indexable and thus not easy to look up.

Anonymous said...

Read Better Builds with Maven.


Maven is not that hard and it is extremely useful as project size grows.

If you think it's too hard then have fun slowly re-writing all of its functionality as you need it.

Unknown said...

@anonymous
I can say that Ant is not that hard, well integrated in eclipse and certainly others, and it is extremely useful as project size grows.

And I don't need to re-write all of Maven's functionality because I don't need all of it. For example, I don't need Maven connecting to internet to do a clean (basically deleting a directory). I don't need Maven trying to replace good and simple Eclipse functionality with a badly written plugin. My list is endless, where is the light in maven land?

Jason van Zyl said...

Then why do you continue to use it? If I honestly had that much trouble from a tool I wouldn't use it. Seriously, use your build script if it's easier for your team. Maybe Maven is just not the tool for you and your team.

Jason van Zyl said...

I'm not trying to be facetious or disrespectful, but trying to wedge a tool into a place where it doesn't fit in your environment doesn't make sense. As Joe Walnes pointed out the other day he likens Maven to Marmite. People generally like it from the start or dislike it form the start. You seem to have nothing but problems so it might not be a good fit.

I'm the first to admit the website is atrocious. I tried to focus on the book (and it's free) as a cohesive reference and we always seem to have contention about the website internally in the project so I just didn't touch it.

I'm also the first to admit that Maven has some problems, but 95% of problems arise from the artifact/repository layer and not-so-great plugins. The artifact layer has been completely gutted in Maven 3.x and around Maven 3.x we will overhaul the core plugins to be small and efficient.

But there are massive changes happening on a daily basis in Maven 3.x and in the m2e integration. Maven 2.x to Maven 3.x will not be like the Maven 1.x to Maven 2.x transition. Almost everything currently working in 2.x will work in 3.x. Even on the 2.x front Brian and John have been pretty aggressive on wiping out regressions and the number of integration tests has been expanded enormously.

We wouldn't be working so hard unless we saw the use. The traffic on central is growing all the time and most clients have user agents and almost all the traffic is from Maven itself. The downloads of Maven increase. The downloads of m2e increase so we're seeing that lots of people are getting some value out of the tools.

You seem to be at a confluence of bad initialization, bad integration, and bugs which makes for an unpleasant experience. People say they don't have problems with their SCM, or IDE but how long did it actually take you to become proficient with those tools? Sure, you should be able to get most things done quickly but to be proficient takes time. I don't think Maven is any different there.

Anonymous said...

It might be worth checking out Gradle (http://www.gradle.org). It has a lot of convention over configuration stuff, but is usually pretty easy to make it do what you want if you already have stuff that doesn't fit the convention. It is also easily extensible and the doco is pretty good too with lots of examples. It is still very much a work in progress, but we use it for large multi-project builds and it works very well. The only downside at the moment is the lack of ide integration, but I'm sure that will get sorted out in time.

Michael Finger said...

This last summer I converted out code base from Ant to Maven. Things were shaky for awhile, but I think everyone is pretty happy now. We have a tooling team dedicated to improving the Maven experience at work. Sure, this is a lot of overhead, but for the size of our development organization the investment is reaping good returns. I could see for smaller companies the benefits vs. costs of Maven could be a lot different, but we got to the point that Ant was not working out anymore.

Some issues that Maven has improved for us are deployments to QA and production, sharing code among teams, keeping track of what libs/binaries projects are using with doing checksums, etc.

Brent Ryan said...

Install nexus maven repository and use netbeans or IntelliJ and your experience with maven will be much better. Eclipse is losing the IDE war now.

Walter Harley said...

@Brent - so, for me to have a good experience with Maven, I need control over every Maven repo I interact with and I need to switch away from the IDE that I prefer?

@Jason - I'll be very interested to see what happens with Maven 3.x. I don't control my project's build infrastructure, other than by collecting and articulating my complaints, so I'm more or less stuck with it.

The thing is, if there is something about our project that makes it a poor fit for Maven, I'm not sure what that is; I think my grievances against Maven would still apply in the context of almost any development process and project that I have experienced, and that's a broad sample. Perhaps the most salient difference is that I do a fair amount of development work in poor connectivity situations - from coffeeshops, airplanes, etc. I think Maven suffers when it does not have a solid, fast connection.

Obviously many people really like Maven! And I respect the effort that has gone and continues to go into it. It's unfortunate that for me it continues to be an obstacle rather than a friend.

Jordan Zimmerman said...

I have no choice to use Maven when I interact with Shindig. I wish I had a choice because I truly hate using Maven.

Anonymous said...

> @Brent - so, for me to have a good
> experience with Maven, I need control
> over every Maven repo I interact with
> and I need to switch away from the IDE
> that I prefer?

No. You need to install Nexus or Artifactory. This will be the only repository you'll have to deal with. Frankly, using tools you don't understand and jumping to all sorts of conclusions based on comments you obviously didn't understand... ignorance compounded, indeed.

Walter Harley said...

@Anonymous: I just poked around a bit on the Nexus and Artifactory web sites. These look like tools that my organization deploys, rather than something that I install locally on my MacBook, which leads me to think that maybe you didn't read my comment that I don't control my project's build infrastructure?

You say that I "obviously didn't understand" the comments about these repo managers. Brent's comment was only that if I installed it my experience would be much better. There is no information in that comment to misunderstand. Perhaps you or he would like to elaborate on why you think using a different repo manager than we currently have would solve the problems I've articulated? (For that matter, what repository manager do you believe we currently have?) For instance, how would deploying Artifactory in our organization make it so that Maven is more resilient against poor connectivity, or that Maven provides more easily accessible help at the command line?

To take a specific example, one problem I have with Maven is that if I change a line of code and recompile, it will check dependencies. If it finds an update it will try to download it, which may provoke additional updates. If, during that process, I lose connectivity, I'm hosed; it doesn't roll back to something out-of-date but workable. (This is one of the problems that I think Maven could solve without needing to change any basic concepts; I want to try to stay constructive here.) Does using Artifactory or Nexus solve that problem, and if so, how? Can I achieve those benefits without changing parts of my company's infrastructure that I do not control?

I named the blog "Ignorance Compounded" because I do indeed think that I'm ignorant, and that most of the people I work with (in the industry overall) are ignorant. Ours is a knowledge industry; there is always more information than can be assimilated, and we are all working in a state of relative ignorance, though some folks are more arrogant than others. As I suggested earlier on, I've got a finite amount of time for learning stuff and an infinite amount of stuff to learn. If a tool requires a lot of tool-specific knowledge in order to use well, it needs to have a correspondingly high payoff. If you're a tool maker, it is not reasonable to expect every user to be an expert; so it is important to ensure that non-experts have a good experience.

Walter Harley said...

@vishwajeet - actually, writing these posts has been very helpful for me; it's helped me articulate, and thus reduce, some of the frustrations I've had.

I'm serious when I say that I keep hoping to see the light with Maven. I do not have the option of just discarding it and moving on, and I do not have the time or the desire to be a Maven expert either. So if I'm to reach a satisfactory middle ground, it'll be in part by organizing my understanding of Maven's limitations and shortcomings, even those that I can't do anything about.

But if that attitude is in turn frustrating for you, I recommend not wasting your time reading this blog :-)

Anonymous said...

My Maven experience.

I started using maven because contracting company we hired insisted on it. They all used IntelliJIDEA, and even tried to convert us (in-house developers) to switch, but I would have to lie to my manager to justify buying it, just to replace Eclipse, which is great tool. Either way, Maven was forced onto us. Buggy Eclipse plug-in required to learn few tricks to actually use it, but in the end, we all wrapped our heads around it. We deployed Artifactory and sort of got control over the whole build process. Now all new projects use Maven as build tool ... not because it is superior, but because we are growing and don't want to mix Maven, Ant and IDE based packaging and library management approaches. So we are sort of stuck with maven.

Was it worth?
Short answer: No.
Long answer: The only maven plugin I am happy to have is Assembly, which when used with profiles proves to be nice solution. Nothing which couldn't be substituted with simple Ant script, but still, like the approach. Apart from that, it is quiet poor experience. Some dependencies force maven to download certain pom files EVERY time the build process starts therefore building offline is impossible. To be honest I haven't dedicated too much time to fix that issue.

So in the end, I have inconvenient build/deployment tool which disrupts my Eclipse IDE experience.

Will I use it future project? I will, but mainly because of peer pressure.

Deron Eriksson said...

10 years ago, I worked at a large tech company in Palo Alto involved in writing a C++-based platform for developing software for scientific instruments. Building our platform was an extremely complicated process that was managed by our extremely talented build guru who had to devote seemingly endless hours to managing our build process. Following this, I worked for over 5 years at a large institution writing Java web applications, primarily involved in portal technology. Here, our builds were at first managed by Ant, and we later migrated to Maven 1 and then to Maven 2.

I was amazed at the simplicity of Ant. In fact, I considered myself fairly proficient at Ant after reading O'Reilly's "Ant: The Definitive Guide" over the course of a weekend. Moving from Ant to Maven 1 was quite confusing since Maven at the time didn't have a lot of documentation, and I found Jelly to be powerful but confusing.

Moving from Maven 1 to Maven 2 was a significant step. Thankfully, documentation had improved. "Better Builds with Maven" was available. (Since then, "Maven: The Definitive Guide" has also appeared.) We had about 20 relatively small but interrelated projects to convert to Maven 2. This process took over a month and at times was difficult. However, when we were done, we had a very elegant project management solution that included being able to build all our projects with one click of a button. With another click, we could generate documentation sites for all of our projects. When things are configured correctly, Maven is remarkably powerful.

We went through some growing pains, but I have to say that after learning Maven 2, I don't think I could ever go back to other tools such as Ant. Maven has a much greater learning curve than Ant, but it also is so much more powerful. I'm sure that many people have situations where Maven isn't the right tool for the job, but I think in most cases, it is a great build/project management tool.

Ideally, Maven would be fully integrated into Eclipse (like CVS is). Currently some great work is being performed in this direction with the m2eclipse plugin.

I like Maven so much, in fact, that I've been working on a Maven repository search engine (http://www.jarvana.com). The quantity of data in the central repository can be overwhelming and the repository can be difficult to browse since it is so large, so I've been writing jarvana.com to try to make it more accessible. Hopefully tools like this can help simplify certain aspects of Maven.

So, personally I feel that although Maven can take a fairly significant investment in time and effort, in general it is definitely worth it when you are done.

Dave Newton said...

> Neither my IDE (Eclipse), my version control system (SVN), my language (Java) nor its libraries were this problematic this far along.

Those are all substantially different entities with a much narrower range of focus than Maven, though.

(And I'd disagree about Eclipse not being as problematic as Maven--I've had far, *far* more issues due to Eclipse or its plugins than Maven.)

@evernat: If your Maven is connecting to the internet to do a clean then it's because you haven't locked down a version somewhere.

Re: Maven overall--sure, there's a learning curve. It's a big tool that does a *lot* of things via its plugins. I haven't had any major issues as I've implemented it incrementally--YMMV.

Walter Harley said...

@Dave - I'm not sure how you can say the Java language, its libraries, or Eclipse has a "much narrower range of focus" than Maven. To the extent that's true, it suggests to me that Maven is trying to do too much.

Re your comment about cleaning: evernat was responding to a comment in my post. I'll say again that I think updating is never the right thing to do in the context of a clean; the only thing updating could achieve is to change the rules of what "clean" means, and the odds that the new rules will apply to an old build output better than the old ones did are in my experience poor. The information about what needs to be cleaned is (in principle, at least) always maximally available from what is currently on my machine; there's no reason I need an update.

That said, I am curious about your comment that it means "a version is not locked down." Can you elaborate? Are you saying that a POM should never refer to a -SNAPSHOT, or are you saying something else?

Very broadly, I feel like I could sum up much of my Maven frustration by saying that Maven seems to assume that the network is fast and reliable and that updated artifacts will reliably be an improvement over what I have already. Neither of those assumptions is valid, in my experience; and more to the point, neither is within my control. So Maven reduces my control over how my day goes - it adds to the likelihood that something outside of my control will render me unable to work. That's frustrating.

unpulped said...

After using eclipse for years - the new company I work with forced me to make the switch to IntelliJ. Maven integrates so much nicer - I refuse to even attempt an eclipse/maven project setup again - we don't have time for those setup/deploy issues.......