A blog and website by Peter Bengtsson

Filtered home page!
Currently only showing blog entries under the category: IssueTrackerProduct. Clear filter

IssueTrackerProduct now officially abandoned

30 March 2012 6 comments   Zope, IssueTrackerProduct

In 2001 I started my first and perhaps most successful Open Source project I've ever made: IssueTrackerProduct. After nearly a decade of maintaining it I have now officially abandoned it.

It all started when I needed a way to track feedback on my personal website. That's why it was originally called "SiteTrackerProduct". I needed something where I could collect bug reports and any other pieces of feedback and then process it in some structured fashion. It was therefore very important that it would be possible to run the application open for anonymous access. People should be able to submit bugs and issues without having to create an account. You see, kids, back in that day it was actually very common that sites would force users to register and create accounts even just because the content owner wanted it. These days, it's common knowledge that to get people to open up and share anything for others to benefit you make it absolutely trivial to jump straight in without having to see a registration page that looks like a tax return form.

Now, since I long ago abandoned the Zope2 application server technology stack and I no longer use IssueTrackerProduct for anything real it's no longer feasible to maintain this project. In the last five years or so we were actually using it actively to track all projects at Fry-IT where I used to work. I have to say, even though we did grow out of it, it was actually successful. It handled the load (after some much needed patches towards optimization) and it was easy for people to actually use since unlike many other bug trackers, it focused on the non-technical end user first and foremost. As much as possible was done to make it trivial to type in your bug or issue and it automatically took care of all notifications and access rights.

Being a personal Open Source project, over the years, it became a melting pot for experimenting and perfecting various new ideas. Many of them we take for granted today but back then it was quite novel if I may say so. This includes:

Writing all of this, I can not resist to get a bit nostalgic. I did sink A LOT of time into this project. Today when I look back at the code and almost feel sick seeing all the mistakes that I made. Much of the ugliness of the code can be attributed partially to the fact that I often used and abused the code to add new features. Also, because we often needed some features (since it was used to manage all of our projects) "yesterday" and then it was hard to justify doing things "properly". For example, the main .py file is over 14,000 lines of code!

I did called it "perhaps most successful Open Source project I've ever made" in the first sentence. The reason for that is that over the years many many people have downloaded it and installed and let it be used by thousands of users. That's something to be proud of.

Anyway! It's time to move on. So long and thank you for all the fish!

The code is still available at

New IssueTrackerProduct release

13 May 2010 1 comment   IssueTrackerProduct

New IssueTrackerProduct releaseToday I finally got around to launching a new version of the IssueTrackerProduct. After a years worth of bug fixes and new features added the most exciting one to me is the Monthly summary feature.

Gosh! I really regret I've never properly collected email addresses for people who actually use the IssueTrackerProduct. I was using a SourceForge mailing list for a while but that got to spam ridden I gave up on it a long time ago.

Check out how long the list of change is on version 0.11

Feedback mucho appreciatado.

Massive improvement on sorting a fat list

28 February 2010 6 comments   Python, Zope, IssueTrackerProduct

IssueTrackerMassContainer is a simple Zope product that is used to put a bunch of IssueTrackerProduct instances into. It doesn't add much apart from a nice looking dashboard that lists all recent issues and then with an AJAX poll it keeps updating automatically.

But what it was doing was it recursively put together all issues across all issue trackers, sorting them and then returning only the first 20. Fine, but once the numbers start to add up it can become a vast sort operation to deal with.

In my local development copy of 814 issues, by the use of pympler and time() I was able to go from 7 Mb taking 2 seconds down to using only 8 Kb and taking 0.05 seconds.

Here's the initial naive version of the code:

    def getRecentIssues(self, since=None, recursive=True, batch_size=20, batch_start=0):
       """ return a list of all the most recent issues """
       issues = self._getAllIssues(self.getRoot())
       if since is not None:
           ... # checking variable since
           issues = [x for x in issues
                     if float(x.getModifyDate()) > since]

       issues.sort(lambda x,y: cmp(y.getModifyDate(), x.getModifyDate()))
       return issues[int(batch_start):int(batch_size)]

So, instead of making a fat list of issue objects, just turn it into a list of the things we really need. The second version:

    def getRecentIssues(self, since=None, recursive=True, batch_size=20, batch_start=0):
       """ return a list of all the most recent issues """
       root = self.getRoot()
       issues = self._getAllIssues(root)
       issues = [(x.getModifyDate(), '/'.join(x.getPhysicalPath())) for x in issues]
       if since is not None:
           ... # checking variable since
           issues = [(t,i) for (t,i) in issues
                     if float(t) > since]

       issue_paths = [x[1] for x in issues[int(batch_start):int(batch_size)]]
       return [root.unrestrictedTraverse(p) for p in issue_paths]

The issue method getModifyDate() returns a Zope DateTime instance which is ridiculously nifty datetime implementation but it sucks in memory use and performance. See this blog about how it sucks compared to mxDateTime and standard lib datetime. So, this time, turn it into a float and then sort. Final version:

    def getRecentIssues(self, since=None, recursive=True, batch_size=20, batch_start=0):
       """ return a list of all the most recent issues """
       root = self.getRoot()
       issues = self._getAllIssues(root)
       issues = [(float(x.getModifyDate()), '/'.join(x.getPhysicalPath())) 
                 for x in issues]
       if since is not None:
           ... # checking variable since
           issues = [(t,i) for (t,i) in issues
                     if t > since]

       issue_paths = [x[1] for x in issues[int(batch_start):int(batch_size)]]
       return [root.unrestrictedTraverse(p) for p in issue_paths]

And the results for my local copy of 818 issues?:

Version 1:
   7842736 bytes (7.5 Mb)
   2.1547999382 seconds

Version 2:
   834880 bytes (0.79 Mb)
   0.210245847702 seconds

Version 3:
   87448 bytes (85 Kb)
   0.0538010597229 seconds

Granted, Zope will release this memory by the garbage collector but why even let it get that big if you have concurrent hits or if anything gets stuck for longer than necessary. Python 2.4 can free memory used but not return it to the operating system to reuse unless the process dies (this was fixed in Python 2.5).

That's a memory usage improvement of about 90 fold and a speed improvement of about 40 fold.

Fry-IT has huge issuetracker instances and at the time of writing keeps 6668 "active" issues across all projects.

Custom Fields in IssueTrackerProduct documentation written

05 June 2009 1 comment   Zope, IssueTrackerProduct

Custom Fields in IssueTrackerProduct documentation writtenThe Custom Fields feature started as a consultancy job in which we agreed the work can be open sourced as part of IssueTrackerProduct so I never got around to write an sensible high level documentation for it. Now I have! From the news piece about it:

"Custom Fields was a feature that was released almost a year ago but didn't have much documentation. Especially easy documentation that describes what it is and how it can be used. That has changed now.

In Custom Fields it is now described what they are and how they can become useful to you. It's such a powerful tool that very few "competing" issue/bug tracking systems can offer."

The written documentation is here: Custom Fields

Feedback appreciated.

IssueTrackerProduct 0.7.2 released

22 May 2006 0 comments   IssueTrackerProduct

A new (development) version of the IssueTrackerProduct has just been released. It contains some crucial bug fixes (to some people) that I really couldn't delay much more.

Sadly I never really had the time to fully implement the TinyMCE support. It's going to come in the next release. It actually works already but you need to know which buttons to press since there is no simple one button to press for it to work.

Helpdeskshow - a quick review

26 April 2006 0 comments   IssueTrackerProduct

I just got back from the Helpdesk & IT Support Show in Olympia (Kensington, London). My main impression is: there are many, big players in this industry.

My pet project, the IssueTrackerProduct is very basic in comparison to some of these companies products. Although it's often used in help desk situations the kind of help desk solutions I've seen today are way different. For many of them, it's all about integrating various systems such as asset management, call logging, configuration management, knowledge management, etc. It seems that the actual help desk apps seems to have to be low priority compared to getting all pieces to fit together.

I walked one lap around the perimeter of the big hall so first I saw the little stands. There they called it "Help desk". As I moved in towards the centre of the hall where the really big players had really big stands, big plasma screens and lightly dressed young ladies a different word was used: "Service management". I talked to a really friendly chap from ICCM Solutions and asked him honestly: what is Service Management? He gave me two answers. One short, one longer. The short version was: the smaller less advanced providers haven't evolved and still call it help desk; the bigger players who have evolved more call it service management. The longer version was that the expression "help desk" is more associated with IT and problems that need immediate solutions. Service management is more about monitoring and controlling: Configs, Incidents, Problems and Change. In that order. It's all business/marketing lingo that don't mean much but at the time he explained it it actually started to make sense. In simplicity, they just want to get away from the word help desk because it's got ingrained annotations to it and they want to do something else.

Very few of the screenshots and demos I saw impressed me in the detail work. I saw endless horrible help desk issue/incident entering screens where some poor user has to select the right fields to fill in amongst a screen packed of different input fields of various sorts. Many systems had issue/incident reports and lists again packed with far too much screen-noise. Some of them had quite a slick and neat design but a lot of them had windows GUI apps that look like a video-rent-shop-epos system from 1995.

Although I never saw it in action, HelpSTAR had (supposedly) a clever solution. You can apparently drag and drop emails from your MS Outlook into their help desk system. That email is then logged, archived and can now be responded to and become a email correspondence ticket. They also had (supposedly) an interesting templating system. That makes it easy to reply to frequently asked questions that people send in to a help desk.

One promising company that I talked to was Logicalware and their product MailManager. Their brochure-work was, unlike all others, really down to earth and easy to read and understand. MailManager is, like the IssueTrackerProduct, written in Zope and is Open Source too. The IssueTrackerProduct has email ticketing capabilities but that's not its core functionality. With MailManager, I think email ticketing is the only thing it does and does it well. Keep up the good work guys!

In conclusion, my IssueTrackerProduct does do much but the few simple things it does it does very well. If you don't have £10,000 to spend on licenses, consultants and headaches and just need a simple system for logging issues, pick my system