TheDarkTrumpet.com

When testing applications, say a contact form emailer, you probably want to display the actual email, and not just some source code of the email. Some reasons why you may want to do this include (but aren’t limited to):

• View the formatting of an email if it’s HTML
• Check colors, fonts, sizes, and the general flow of the email
• Maybe more comfortable with that way of looking at it (through an email client)
• Want to compare more accurately two different emails
• Want to make sure emails actually send…

Here’s the problem…

Lets say you’re working on a number of projects, and you have your test data but some of that test data may have real emails – say to end users. The worst thing you want to do is to send a test email to a user when they aren’t expecting it. To get their opinion is important, so not disabling mail totally may be important.

Because recently I came across this issue, I wanted to come up with some solutions – more language nonspecific ones to handle the situation.

Option 1:
You can have a variable set that denotes your system is a test system, and doesn’t send the email. Different frameworks handle this differently. One internal to my current job did this – and instead, stuck the email in a log that was in the database. Other frameworks stick this email in a buffer that you can view later. This definitely is a solution, and works pretty well, but not perfect in my opinion. The reason being is you don’t really see the email as the end user would see it. You know it exists, and that works to a degree, but outside that…you don’t see that much.

Option 2:
Have each of the emails going to stdout. So when you send an email, it outputs to the terminal. This suffers the same issues as Option 1, and is done in some frameworks. This is better than option 1 though, because it’s easily able to be seen. No querying of special output buffers, no checking the database, it’s all right there. The problem this suffers mostly with is with say a thousand emails, this doesn’t scale. This is currently a debate on the django project, since it doesn’t support either of these features in a development environment. Read more about it here:
http://code.djangoproject.com/ticket/8638

Option 3:
The third option is just to make sure each of your emails in fixtures is to your own, or a test. Unfortunately while this solves the problems for option 1 and 2, you can make a mistake and something gets sent. Furthermore, if you’re commenting out code on emailing the business owner then you’re really doing it wrong in my opinion. For testing, your code really shouldn’t change.

Option 4:
This option came up as being the most benefit for myself, as of last night (having thought about this for while). Many know I run my development environment in a virtual machine – not on the base system. So what I did was install postfix, and disable the email sending outbound (local accounts will work). Furthermore, this solves all the above problems!. Your code isn’t changing, and it actually gets to sendmail – where we are making a decision to trust sendmail (or postfix) at this point. Personally I’m OK with that…anyways I feel this allows you to fully test emails since you can either tail the email log for when they come in (like some frameworks going to stdout basically do), or you can test in an email viewer quite easily. I really like this option, although implementing it was a bit of a pain.

I want to discuss the steps I went through to make this work for me. Unfortunately, I’m not a postfix expert, and there are a few steps I’m not sure why were needed. Especially the /etc/aliases file needing to have a map to myself…so if anyone can see improvements on this, please comment and I will add your changes and give you credit.

Here are the steps:
Postfix Setup (for generic emails):
1. Install postfix
2. Edit /etc/postfix/main.cf
— mail_spool_directory = /your/spool/directory
— Comment out ‘home_mailbox = .maildir/’
— alias_maps = hash:/etc/aliases
3. Edit /etc/aliases
— Add an alias, e.g. dthole:dthole
— run ‘postalias /etc/aliases’
4. Disable smtp out and in.
— Edit /etc/postfix/master.cf
— Comment out both lines that have smpt in the beginning.
5. Add an always_bcc
— Reedit /etc/postfix/main.cf
— Add always_bcc=yourname@localhost

What the above actually do is force sendmail not to actually send anything out, but the always_bcc local will always deliver it to youname@localhost. This email won’t be a bounce, but will be the actual email itself. From there, just setup your Thunderbird email client, or whatever, to access the unix mail file. If you use virtual machines, you can either create a simlink for this, or just make maildir your home directory (basically re-commenting it out).

If there are other ideas, I’ll post them below as well.

I did quite a bit of discussion on IRC about this with some postfix geeks before posting, and one option another mentioned was the use of defer_transports, to throttle it off. I’m not sure how that works, and the documentation didn’t help the most so I’ll let that up to a commenter to give more details.

Anyone’s who’s used rails already knows about helper functions. Basically, for those that don’t, they are functions you call in your views to output HTML. My question is, why!?

Take for example link_to (google to find out more, railsapi, which I just tried going to is giving an application error). In it you have the format of something like:

link_to “link name”, :controller_name, …

What does it output? an a-href tag linking to the controller. Now, some people may be thinking “hey this is cool” or “hey, I don’t see the problem”, but I’m kinda curious why we needed to redefine the a-href link. Some may say, so we don’t link to the specific controller (e.g. typing in /foo, which would be not DRY compliant) – in which case, they are exactly right. What I’m curious about though? Why not do it the way Django does it?

Take this example:
{% url myview.method id %}

now, you can also do it by names, but instead I’ll give the documentation for full reference:
http://docs.djangoproject.com/en/dev/ref/templates/builtins/?from=olddocs#url

Why do I like this? It returns something like:

/foo/1, if the ID is 1, and foo is your view method.

Why do I like this better? Because it’s not attempting to redefine HTML, it returns the link, without redefining the HTML that goes with it. I don’t exactly understand why Rails decides it’s important to redefine the way a link works in this fashion.

I will give Rails the benefit of the doubt in that they probably thought this was a good idea and aren’t changing it to keep backwards compatibility. There are a number of helper methods, though – maybe I’m just not getting it, or maybe I’m just not cool enough to like everything about rails. I’m sure it’s a matter of time before I get someone flaming at me on my post here I definitely don’t feel any framework is the best – although I will admit with the Rails people I know, it’s almost always touted as being the definitive trophy that everyone should strive to reach. Anyone who doesn’t go for this trophy is foolish and wrong.

sigh

Edit: While mentioning this to a friend, I recalled what one person told me as the reason for the link_to. The reason was for the DRY principle. This basically means: Don’t Repeat Yourself. This is cool in all, but if you want 5 links, you’re going to do 5x a hrefs, or 5x link_to – the same basic stuff. If you want it in a loop of link_tos, you can do the same with a loop of ahrefs in Django. I don’t think the link_to really saves anything.

Managing code, projects – deadlines, tasks, and integration with a team is a tough task. A lot of people, and businesses I feel could be doing this a lot better. It starts with the single person actually, that will change your – and eventually your team’s way of how to manage your “stuff”. I’m a bit torn on putting this only in development – as it’s both development and personal productivity.

Lets begin with the first problem. Source code management – but it’s not only source code that needs managing! Every bit of information that you routinely change can benefit from a management system. Personally I used to use SVN for a lot of stuff. SVN is pretty fantastic in that you can keep a log of your entire development history. Source code management is definitely not only for teams. Every project I have done in the last 3-4 years is versioned, everything I start is versioned and has a specific location and setup associated with it. I’ll get to the structure of projects later. Instead of SVN, I personally now recommend git – and I don’t mean getting a graphical-type app to interface with GIT – I mean, learn it from the command line. GIT is a lot stronger than subversion in my opinion, and has better perks for single developers because creating a project is as simple as going into the directory and typing git-init. I’ll try to provide some resources a bit later.

Now for the next problem, that being documentation. Even for personal projects, it’s very very important to document your code. Why? I can’t tell you how often I go back to code I wrote and wonder why I did it the way I did. A lot of it is growing as a developer, but sometimes there’s a darn good reason why I did it a certain way and in the process of “fixing” what I thought was wrong – I actually create more problems. Every single project, little or large should have some documentation. By my own preference, I begin the documentation phase well before I even write a line of code. I come up with brainstorming charts, flow charts of some ideas, and pages that I reference to learn about what I’m trying to fix. Why not just start coding? I can’t tell people how often I messed up and began coding only to find out a solution existed and was simple enough. Also, when you plan out things ahead of time, you find out that perhaps it’s either more complicated or more simple – and your perspective and plan for what to do changes. For anyone who took the 7-habits training, this is basically the most important stage focused in a lot – instead of just reacting.

Structure is the third really important area. Your code, and documentation need to have “standards” for where they are at. Why? I’m sure the search feature on a computer is fantastic, but knowing where to look is easier than searching. For personal programming projects, I tend to keep the code and the project documentation together. I use the directory structure below:
/code
/documentation
/templates [for django apps]
In the code section is the main project files – this isn’t very language specific – just whatever you code goes here. The documentation is where everything for documentation goes, including documentation generated from code (e.g. pydoc, phpdoc, etc). Templates is also very simple, for all my django apps I use a templates directory to keep my own themes outside the built-in themes of a django app.

What about managing your tasks related to a project? There are a LOT of options available for this. Personally, I find myself using a project called Trac. There’s no real specific reason why I use this except I like the way it is presented, gives me graphs of how far along a project is, timelines, and so on. It also integrates with GIT pretty well, so seeing what changed, GUI style, is super simple (I tend not to use this, but it’s there). There are a number of projects that exist, but I recommend setting up one. The reason why they are helpful is because you can split out your brainstorming and “needed ideas” away from the tasks you’re doing. For example, when I’m coming up with ideas for a project i’ll place them into Trac as “wants” for version 1 (I call them “release 1″, “release 2″, etc – about once a month time). When I begin working on version 1, I put new stuff that aren’t bugs into version 2. When working on version 2, put them into version 3. The general idea is to avoid your own scope creep by realizing that you have your own internal deadlines that you want to make – and adding “hey this will be cool” to it is just going to make the deployment longer.

Well, there you have it, these are the main building blocks that I use to actually get projects completed. Is this a definitive list? Definitely not. I missed a number of fine grain details including the whole general GTD thing (which is outside this). A lot of people may be thinking that this is fairly complicated to follow and doesn’t have much value – but I beg to differ on that. While, in general, you may lose some time at first on projects, in the end you’ll gain better projects. Ideas that you come up with and document will end up in the project, where they may end up being lost because you didn’t note it down. You’ll also get the wonderful joy of seeing these little bars fill up as you get further in your goals. So all this gives a nice self esteem bonus as well. After a bit of time, the whole process will feel like second nature.

Resources:
Trac: http://trac.edgewall.org/
GIT: http://git.or.cz/
GIT from the bottom up (my favorite ebook on this version control system): http://www.newartisans.com/blog_files/git.from.bottom.up.php

In case you haven’t heard the news yet, Django version 1.0 has been released last night at 7 PM. It’s a few weeks further than they anticipated, but darn close to perfect on schedule. There’s now a focus more on backwards compatibility – which is fantastic. Once all the stuff on django pluggables is updated to use 1.0 then they should be more stable in the long run.

If you haven’t used Django since .96 or so, there are a whole lot of new features and a few things that were removed. Migrating code to the newer version may be a bit painful, but is worth it.

The official story is here:
http://www.djangoproject.com/weblog/2008/sep/03/1/

Very awesome job on completing version 1.0!