TheDarkTrumpet.com

For pretty much my entire professional career, I’ve been stuck doing pretty much debugging. While others have been given the opportunity to create and all, I’ve been more trying to fix messes people create. This gave me maybe a bit more of a conservative approach to programming than my peers…but that’s for another blog entry perhaps. What I want to descrive here is what I’ve learned in the years I’ve been stuck trying to fix code.

First of all, I want to say that programming is hard – but debugging stuff is harder. There are tricks – but the first and most important trick is leave your laziness at the door, you won’t survive debugging much unless you do so.

So the tips:

/Don’t be afraid to get your hands dirty/: You can learn a whole lot by breaking something and then repairing it. This doesn’t mean go about breaking code on a production system, but if you have your own sandbox that isn’t gonna be hurt screwing up – just mess around. The more you screw up stuff, the better you’ll learn.

/Don’t be afraid to take time in figuring stuff out/: Really when you’re debugging, don’t expect that the solution will be totally obvious or that you’ll get it done right away. Some issues I’ve worked with have taken near a week or so + of a little work every day to finally accomplish fixing. You’re unlikely to be timed on how fast you can squash bugs – and you’ll reduce your stress level a whole lot by just chillin and spending a bit of time here and there.

/Keep a notepad nearby/: I can’t begin to emphasize how important this is. Your notepad is basically your breadcrumbs. You need to jot down notes, flow charts, variables where they are at various stages of the program, and so on. Your notepad should be your #1 tool, it’s way more imporatant than any technological tool you feel you need to accomplish the task.

/Know how to follow the electronic breadcrumbs/: var_dumps, echos, prints, puts, and so on – these are your keys to where things are going. Some languages/frameworks have really good debuggers, but again you’re relying on tools. Sometimes the best thing you can do is just output something where it’s visible and take a look at it.

/Be sure to read the entire problem reported/: A lot of times I’ve found that if I skip over a bit of a ticket, I waste more time in the long run because I thought I knew better or may have read it already. This reminds me of an activity when I was a kid. On the top of the page it said to read everything before beginning – and like many impatient people (which was most of my class), I started right away. The last question was to skip the first 20 steps and do the first and last two. I felt like an idiot, and as you can tell, this lesson has stuck with me for a long long time.

/Know your resources/: Generally speaking, I hate asking people for help. I’m probably polar opposite of everyone else in the office in this regard (and perhaps in other regards too), but I prefer to rely on myself. With that in mind, I do a great deal of research on my own before I go about asking anyone anything. While you may not want to go as far as I, don’t make the human next to you or in your office your first go-to for the answer. Do some research, spend some time looking it up. Then, if you go up to the person for help in the end at least the person will be impressed by the research you attempted to try beforehand. This is just like math, know your resources and really try at problems before having the teacher give you the answer. With this in mind, google is probably your favorite friend. You’ll never know what you get out of it.

/Remember balance/: Balance in everything in life is important, and it’s no different here. Bug squashing all day long is enough to drive anyone senile (or at least make someone really on edge). Try to mix in some stuff in the day – such as tools to make one’s life easier, or innovate a bit more here and there, maybe refactor something, or document something.

In the end, really all this comes down to as few of tools as possible to get the job done and you getting in the engine and working on it. Remember that if you’re debugging something, your job is unlikely to be making things perfect. Try to get the engine working, and be happy with that. Perfection in this world is pretty much impossible to obtain.

I hope this helps some people.

“We can thus say that anger is caused by the discrepancy between what we expect and what we get.” – Source

This is a very similar quote that goes through my head that I heard elsewhere. The general idea is that when we expect something, and what we get doesn’t match up with what we want – we feel anger. Anger isn’t the only emotion that suffers this, though. Frustration, is very similar to anger in this sense. I’ve met a number of programmers who meet a situation in coding where the code they are looking at is not as they would have wrote it – they may not even understand it – but after some time (maybe even hours), they feel frustrated. Frustration is very common with large problems, but they don’t need to be.

First a bit of a story. I’m working at my 2nd full time job right now. In my first job, I worked with PHP code in a enterprise environment. My job was to maintain, fix, diagnose, enhance, and so on – the centerpiece of the main application. The only other person working on this was the lead developer. In an effort not to bother him too much – I took it upon myself to try to spend a lot of time trying to learn more about their programmer. In my 2nd job, the lead developer left, leaving a very difficult and complex project needing work done – so adjusting is the name of the game.

I want to go over a few different ways that I went about learning to debug, as some others coined – flashlight debugging. Generally speaking when you debug something you need to remember that while tools are just that – tools, don’t make it a crutch. A lot of my ideas are very very language inspecific and should hopefully make sense.

• Pretend you’re the only developer – It’s a hard thing sometimes not to want to just go up and ask for help on something right off the bat. The problem with this is that we need to be able to strugle about a problem for awhile to really understand it. Remember, you have the internet, books, newsgroups, and so on to help you out when researching it. The key part is help from another human – either through IRC, chat, or whatever..best to use it last, even if it saves time.
• Debug in your one place, and spread out – The problem with very large systems is trying to understand too much at one time. Some people feel it’s important to learn the entire system and then fix the specific problem. The problem with this methodology is that the system may be too complicated, and too little documented to bother doing that. Instead, focus on your one small area and spread out from there.
• Be sure to research

  - Check existing tickets in your ticket management system, look at the log of the file, who edited it and why (subversion, or whatever), read wiki articles on the business process, and learn about the problem you’re solving and not just the problem itself.

• Diagram and keep your own documentation – Whenever I start a problem that I feel will take a bit of doing, I usually will either use graph paper, or a whiteboard to diagram out my thoughts and needs. The key of this is don’t be afraid to go to multiple files, write stuff out, go to others write those out, and link them together. The diagrams may not be of use to others, but will be helpful for you. For your own documentation, I highly recommend your own wiki. Personally, I use emacs wiki, which is fantastic for keeping personal documents in. It’s easy and fast to come up with an idea, and note it down there as well. It’s an extra place to search, but it’s your own library. Little notes on links and ideas you find on the net can go in this as well.

The part that’s really important to keep in mind is that the world is rarely perfect, and neither is code. It’s easyto get trapped in the idea that code should be perfect, either in coding style or whatever. For some ideas I think that may help groups out are:

• Documentation in Code:

  Documentation is a tricky thing since too much documenting and it’s going to make the code harder to read. Too little documentation, and that’s not really good either. The “perfect amount” I’ve found is only in areas where you see that if you came back a month later that you may not understand what was done. Keep them simple, keep them short though.

• Document in SVN:

  Subversion is more than just a place to track changes, but also commenting what issue/bug/defect/feature you fixed/implemented will go a long ways toward making what you change match up with the official logs.

One last bit of advice I want to make is to approach solving a computer programming problem much like you’d solve a math problem. With a clear head, calm and collected – patience and diligent, write out the problem. Don’t be afraid of making mistakes, create new projects to test ideas and integrate them in. Break stuff, fix it up again, and break it again. Lots of learning comes from fixing stuff, and making mistakes. Don’t be afraid of making mistakes, use it as a learning opportunity.

Edit: I wanted to add a few resources that will possibly help out a bit.

• 43 Folders (Blog) http://www.43folders.com/ Basic time management, definitely worthwhile to learn about.
• Chapters 2 – 4 in The Art of Software Modeling – This book is fairly good from a system analysis standpoint to learn tips on how to debug code or to learn very complex systems. I’ve used nearly every method they described, and it works well. I don’t cover most of that information here, but the book is available on Books 24×7 to those who have access to it.

I wanted to discuss how to create a login-type app today. I’m working on a project for work and came across the need for a login page (well, and logout too). Now, while I may change this to the built-in method, I wanted to share the knowledge I gained in dealing with some oddities of the forms setup in general.

When you’re creating a login form, it’s generally best in my opinion to use the forms portion of Django instead of straight calls. It’s more DRY compliant, and you have a lot of stuff taken care of for you when it comes to the output. The problem I came across with was about data as it’s sent back.

So…I won’t concentrate much on how to use the forms setup (maybe I should create a tutorial maybe), but your general authentication setup may look something like mine:

def auth(request):
if request.method == ‘POST’:
form = LoginForm(request.POST, error_class=LoginErrorOverride)
if form.is_valid():
username = form.cleaned_data['username']
password = form.cleaned_data['password']
user = authenticate(username=username, password=password)
if user is not None:
if user.is_active:
login(request, user)
return render_to_response(‘auth/loginsuccessful.html’)
else:
return render_to_response(‘auth/logindisabled.html’)
else:
form.errors['login'] = ‘unsuccessful’ #Adds an error, so we can raise it later.
form.data = {‘username’: form.data['username']} #Clear our data, except for the username.
else:
logout(request)
form = LoginForm()
return render_to_response(‘auth/login.html’, {‘form’: form})

So this basically tries to keep as little logic as possible needed for the forms. I create the object from my forms class, check validation, then check login. If it’s invalid, or if it’s an incorrect login, I want to redisplay the login web page with giving them a try again.

The problem I had was that reloading the page meant that the password was entered as a “default field”. Now…in my opinion, the password widget really should somehow override this behavior because it’s absolutely stupid. For the most part, password fields should really never be outputted back to the user. That’s where the line:

form.data = {‘username’: form.data['username']} #Clear our data, except for the username.

takes care of the situation. I’m not fond of this solution, but it does work. The optimal solution is that for a password widget, we should have it automatically clear the data for us.

When testing applications, say a contact form emailer, you probably want to display the actual email, and not just some source code of the email. Some reasons why you may want to do this include (but aren’t limited to):

• View the formatting of an email if it’s HTML
• Check colors, fonts, sizes, and the general flow of the email
• Maybe more comfortable with that way of looking at it (through an email client)
• Want to compare more accurately two different emails
• Want to make sure emails actually send…

Here’s the problem…

Lets say you’re working on a number of projects, and you have your test data but some of that test data may have real emails – say to end users. The worst thing you want to do is to send a test email to a user when they aren’t expecting it. To get their opinion is important, so not disabling mail totally may be important.

Because recently I came across this issue, I wanted to come up with some solutions – more language nonspecific ones to handle the situation.

Option 1:
You can have a variable set that denotes your system is a test system, and doesn’t send the email. Different frameworks handle this differently. One internal to my current job did this – and instead, stuck the email in a log that was in the database. Other frameworks stick this email in a buffer that you can view later. This definitely is a solution, and works pretty well, but not perfect in my opinion. The reason being is you don’t really see the email as the end user would see it. You know it exists, and that works to a degree, but outside that…you don’t see that much.

Option 2:
Have each of the emails going to stdout. So when you send an email, it outputs to the terminal. This suffers the same issues as Option 1, and is done in some frameworks. This is better than option 1 though, because it’s easily able to be seen. No querying of special output buffers, no checking the database, it’s all right there. The problem this suffers mostly with is with say a thousand emails, this doesn’t scale. This is currently a debate on the django project, since it doesn’t support either of these features in a development environment. Read more about it here:
http://code.djangoproject.com/ticket/8638

Option 3:
The third option is just to make sure each of your emails in fixtures is to your own, or a test. Unfortunately while this solves the problems for option 1 and 2, you can make a mistake and something gets sent. Furthermore, if you’re commenting out code on emailing the business owner then you’re really doing it wrong in my opinion. For testing, your code really shouldn’t change.

Option 4:
This option came up as being the most benefit for myself, as of last night (having thought about this for while). Many know I run my development environment in a virtual machine – not on the base system. So what I did was install postfix, and disable the email sending outbound (local accounts will work). Furthermore, this solves all the above problems!. Your code isn’t changing, and it actually gets to sendmail – where we are making a decision to trust sendmail (or postfix) at this point. Personally I’m OK with that…anyways I feel this allows you to fully test emails since you can either tail the email log for when they come in (like some frameworks going to stdout basically do), or you can test in an email viewer quite easily. I really like this option, although implementing it was a bit of a pain.

I want to discuss the steps I went through to make this work for me. Unfortunately, I’m not a postfix expert, and there are a few steps I’m not sure why were needed. Especially the /etc/aliases file needing to have a map to myself…so if anyone can see improvements on this, please comment and I will add your changes and give you credit.

Here are the steps:
Postfix Setup (for generic emails):
1. Install postfix
2. Edit /etc/postfix/main.cf
— mail_spool_directory = /your/spool/directory
— Comment out ‘home_mailbox = .maildir/’
— alias_maps = hash:/etc/aliases
3. Edit /etc/aliases
— Add an alias, e.g. dthole:dthole
— run ‘postalias /etc/aliases’
4. Disable smtp out and in.
— Edit /etc/postfix/master.cf
— Comment out both lines that have smpt in the beginning.
5. Add an always_bcc
— Reedit /etc/postfix/main.cf
— Add always_bcc=yourname@localhost

What the above actually do is force sendmail not to actually send anything out, but the always_bcc local will always deliver it to youname@localhost. This email won’t be a bounce, but will be the actual email itself. From there, just setup your Thunderbird email client, or whatever, to access the unix mail file. If you use virtual machines, you can either create a simlink for this, or just make maildir your home directory (basically re-commenting it out).

If there are other ideas, I’ll post them below as well.

I did quite a bit of discussion on IRC about this with some postfix geeks before posting, and one option another mentioned was the use of defer_transports, to throttle it off. I’m not sure how that works, and the documentation didn’t help the most so I’ll let that up to a commenter to give more details.

Anyone’s who’s used rails already knows about helper functions. Basically, for those that don’t, they are functions you call in your views to output HTML. My question is, why!?

Take for example link_to (google to find out more, railsapi, which I just tried going to is giving an application error). In it you have the format of something like:

link_to “link name”, :controller_name, …

What does it output? an a-href tag linking to the controller. Now, some people may be thinking “hey this is cool” or “hey, I don’t see the problem”, but I’m kinda curious why we needed to redefine the a-href link. Some may say, so we don’t link to the specific controller (e.g. typing in /foo, which would be not DRY compliant) – in which case, they are exactly right. What I’m curious about though? Why not do it the way Django does it?

Take this example:
{% url myview.method id %}

now, you can also do it by names, but instead I’ll give the documentation for full reference:
http://docs.djangoproject.com/en/dev/ref/templates/builtins/?from=olddocs#url

Why do I like this? It returns something like:

/foo/1, if the ID is 1, and foo is your view method.

Why do I like this better? Because it’s not attempting to redefine HTML, it returns the link, without redefining the HTML that goes with it. I don’t exactly understand why Rails decides it’s important to redefine the way a link works in this fashion.

I will give Rails the benefit of the doubt in that they probably thought this was a good idea and aren’t changing it to keep backwards compatibility. There are a number of helper methods, though – maybe I’m just not getting it, or maybe I’m just not cool enough to like everything about rails. I’m sure it’s a matter of time before I get someone flaming at me on my post here I definitely don’t feel any framework is the best – although I will admit with the Rails people I know, it’s almost always touted as being the definitive trophy that everyone should strive to reach. Anyone who doesn’t go for this trophy is foolish and wrong.

sigh

Edit: While mentioning this to a friend, I recalled what one person told me as the reason for the link_to. The reason was for the DRY principle. This basically means: Don’t Repeat Yourself. This is cool in all, but if you want 5 links, you’re going to do 5x a hrefs, or 5x link_to – the same basic stuff. If you want it in a loop of link_tos, you can do the same with a loop of ahrefs in Django. I don’t think the link_to really saves anything.