To be fair, it’s not your fault. This is the way most people handle email. To ensure that those important emails get to the inbox, there is a lot more involved. I’m going to explain how to make it happen in two ways: The hard way and the easy way.

Before getting into the details, I should probably tell you about myself. My company, Wildbit, runs Postmark, which is an API based service for delivering email in web apps. While Postmark is new, we’ve been running Newsberry (an email marketing system) for years, giving us the experience to properly handle a large volume of email for many customers. Let’s jump in.

The Hard Way (Brace Yourself)

Email is old and seemingly simple, while at the same time extremely effective. If you send even a moderate amount of email from your app, you can outsource the complexities to a provider like us, or do it on your own. Here’s what you’ll need to know if you want to do it yourself.

Mail Servers and DNS

Obviously to send email, you need a good mail server. Any well known open source email server will work fine. On the commercial side there are some very good providers that offer more specific configuration options for outbound sending. Personally, I highly recommend PowerMTA from Port25.

The next important step is configuring DNS (Ed: Confused by DNS? Read this Think Vitamin article by Chris Lea). Without this, receiving mail servers have no idea who you are. Make sure to give your mail server a dedicated IP address and a domain. Now, this is where most people stop, and also where they get into trouble.

The final step is having a proper PTR (or reverse DNS) record for the dedicated IP address. Most email receivers require that your IP address resolves to the domain you chose for your mail server. Without this, many ISPs will quickly block your messages.

Email Authentication

Once your mail server is up and running and messages are sending, it’s time to think about authentication. No, I am not talking about a username and password. Email authentication was created to help validate and verify senders. The standards we’ll discuss are SPF and DKIM

SPF (Sender Policy Framework)

SPF is pretty simple. You create a record in your DNS and define the IP addresses or domains who are allowed to send for your domain. When an email is sent, the receiving mail server looks this up and decides if it should accept the message.

DKIM (DomainKeys Identified Mail)

DKIM is also managed via DNS, but is very different. With DKIM you designate the various servers that you send email from (email marketing, transactional, departments, etc) and give each one a private and public key pair.

The public key is added to DNS and the private key is configured on your mail servers. When you send email, each message is “signed” with the key and used by receiving mail servers to identify and verify your domain.

Reputation & Monitoring

If you send even a moderate volume of email, you can be sure that ISPs are building a reputation on you. They watch email coming from your IP or domain, then track things like spam complaints, bounces, unsubscribes and volume. From there, they can determine a reputation and decide how to treat your messages (inbox, reject, spam).

ISPs are now building reputations on the from domain, and further, basing that reputation on things like opens and clicks (engagement). Remember that DKIM stuff? By signing your messages, it helps email receivers identify your sending domain and build a good (or bad) reputation that you can hold on to.

While ISPs don’t publish their reputation data, there are two well known public resources: Senderscore and Senderbase. You can use these for free to track your own IP reputation, but keep in mind they are not definitive and should be used for reference only. The best way to test delivery results is through something like Mailbox Monitor from Return Path, which will track the inbox, missing and spam rates of a seed list to major ISPs worldwide.

Throttling (or Being Polite)

The amount of spam ISPs need to deal with is both overwhelming and expensive. As a reputable sender, it is important to remain polite and follow their rules. Each ISP has rules about how you should send email to their servers. This includes the number of connections you can open, messages per connection, what to do if you receive certain responses.

Remember I mentioned commercial mail servers? This is where they really shine. PowerMTA, for example, has some great options to not only control these settings, but to change them based on the responses from the ISP. For example, during the holidays many email servers get overloaded and start rejecting connections. Instead of overloading them more and reconnecting, the mail server can backoff a bit to reduce the load.

Whitelists and Feedback Loops

Heard enough already? I’m almost done so you can read the easy way. The final important aspect of sending email is getting listed (no, not blacklisted). A growing number of ISPs having something called a Feedback loop. In short, it is a way to get notified when someone clicks “This is spam” in their email client.

Most people don’t even know about it, but it’s just as important as having an unsubscribe option. By tracking spam complaints, you can easily remove these people from your list and avoid damaging your reputation. This is also a good metric to understand how relevant your email is to subscribers. Something to keep in mind, AOL (and others) consider 1 complaint for every 1,000 messages a target rate for bulk senders.

Whitelists still exist, but only with ISPs such as Yahoo, AOL, Verizon, and United Online. If you send a high volume of email, you may want to look into commercial accreditation services like SenderScore Certified and ISIPP.

Okay, even I am bored at this point. Almost all of the stuff I just told you can be accomplished with hardly any effort, so let’s jump to the easy way.

Doing it the Easy Way

An easy way actually exists, and all you have to do is use a service that already has all of this in place and follow their instructions. When it comes to sending transactional email from your web app, there are a few services popping up lately, but I’ll just explain how we tackle the above in Postmark.

Mail Server and DNS

When you create an account, you can add a “virtual server” for each project or site. In the background, all of the magic for configuring the mail server and setting DNS is done automatically. All you have to do is drop some code into your app (we have many libraries) or point to our SMTP server to start sending.

Email Authentication

Before you can start sending, Postmark requires that you create a “sender signature” for each FROM email. This confirms that you own the email address, but it also informs us of the domain you use for sending. By knowing the domain, it is then easy to create a DKIM pair and explain the proper settings to add DKIM and SPF to your DNS. It turns a complex process into something easy and straight forward. Even if you choose not to implement DKIM for your own domain, Postmark will still sign emails with our own domain, allowing you to “borrow” our reputation.

Reputation and Monitoring

By using an Email Service Provider (ESP), you are also using their reputation. It’s extremely important to make sure that the ESP handles abuse before it happens and keeps their customers under control. By maintaining a good reputation across a high volume of email, you then benefit from the reputation for your own messages.

With Postmark, since we sign outgoing messages (DKIM), you can either benefit from building your own reputation or piggy back on our well maintained reputation.

When it comes to tools, we use ReturnPath and ISIPP to monitor our reputation, inbox rates, and blacklists. If any issues arise, they are known immediately so they can be resolved. Of course, maintaining a good reputation is all about stopping things “before” they happen. This is why we have our own algorithms in place to alert if a customer is misbehaving. This could range from high spam complaints to content issues or even anomalies in volume.

Throttling

By using PowerMTA, we can quickly analyze and adapt to what the ISPs want when it comes to sending practices. These rules can change based on reputation as well, so they go hand in hand. For instance, if we add a new IP address to our servers it is basically unknown to the ISPs. We can easily “warm up” this new IP address slowly over time until it has built enough of a reputation to send a higher volume.

Whitelists and FBLs

Out of everything mentioned, this is the easiest part. For each IP or sending domain we just make sure we have feedback loops in place to monitor your spam complaints and maintain any existing whitelists. An elegant reporting interface is available for customers to view spam complaints if they occur.

Wait, There’s More!

There are some features that a hosted provider can offer that a regular mail server might not provide. The most important is bounce handling. By using Postmark, all bounce types are automatically tracked and if appropriate, deactivated. For instance, if an email no longer exists you should stop sending to it.

This is not only polite to the ISP, but also helps you recover and report these problems to your customers or members. You can also categorize (tag) and track messages that were sent, along with a full archive of the content. These features are great for troubleshooting and monitoring.

Summing it Up

If you are still with me, the short answer to Email Delivery for Web Apps is just like anything else: Focus on what you do best and give the rest to the experts. By using a hosted service for email delivery you can have more confidence that your emails are delivered to the inbox.

Further reading and resources:

• Postmark
• Sender Policy Framework
• DKIM and DKIM Core
• MAAWG Senders Best Communications Practices
• ISP Information on Word to the Wise
• Return Path
• ISIPP
• Port25

Questions or Feedback?

If you have any general questions about email, deliverability, or want to discuss, leave a comment below or email me at cn@wildbit.com.

Before writing this article, I had not given much thought to exactly how we work in a virtual environment. My first answer was simply practice and many mistakes. Although, the knowledge gained from those mistakes can be narrowed down to three main ingredients: The people on the team, the process that drives the team, and clear communication.

The people

Skip the resumes

You can have a good process and great tools, but it all starts with a team who can work well together. I have been running Wildbit for almost eight years. In that time, I learned to not hire just based on need or skill set, but to hire based on interests, personality, and a motivation to constantly improve. Hiring virtually makes this much harder. You can’t sit down and feel out a person or get to know them, you need to base their personality and passion other factors. For instance, we usually ask people what books they read, blogs they follow, or people they look up to in the industry. This helps us learn about their interests, but can also demonstrate their passion and dedication to their work. If people take time to write a blog, it tells me they are confident in their ideas and willing to work on their own. The key is to find people who love what they do, then provide the environment for them to express it.

Plan for mistakes

If you’re planning to hire someone remotely, be prepared for mistakes. Most people would love to work from home, but most people are also not cut out for it. We’ve had situations where very talented team members did not work out. Not due to their skills or interest, but their inability to work from home without supervision. Once this is realized, the process of trying to supervise and recover can be toxic. The manager tries to improve the situation while the employee has a constant feeling of failure. In these scenarios it’s very important to trust your gut and move on. In a virtual team, always make sure to plan for these mistakes and accept delays and cost.

Listen and learn

There is a psychological element to running virtual teams that is much different from the physical environment. Instead of reading body language, we have to pay close attention to habits and read between the lines. I can quickly notice from an IM conversation when someone is not motivated or is losing interest. Even the smallest things like emoticons, punctuation, and frequency of participation can reveal a lot. The trick is to know each person, their usual habits, and when they start changing those habits. It’s pretty easy for someone to hide mistakes or poor productivity. By encouraging open and honest communication, the issues are most likely to surface sooner.

The process

If you hire the right people who enjoy what they do, most of the painful management issues go away. At the same time, a solid process is still necessary to improve productivity. A specific process should always be tailored to the needs of your company, but I can give some insight on what works for Wildbit.

Focus on results, not time

We’re a software company, so our process is specific. We work strictly in weekly iterations. Each week, each person is dedicated to one project. This clarifies communication across the team, sets expectations, and maintains focus. Everyone on the team always know what each person is working on, as well as our clients.

Weekly iterations also help us avoid hour tracking and baby sitting. Each week is planned based on a desired outcome, not the number of hours worked. This avoids tracking and posting hours and enables us to deliver results. At the end of the week, it is clear to everyone on the team if the expected results were delivered.

Schedules matter

A primary advantage to working remotely is a flexible schedule. While we don’t have rigid daily working hours, it is still important that each person has a consistent schedule. We try to have a range of time when each person will be available, with a plus or minus flexibility. This sets clear expectations across the team with some leniency to start work when the person is ready. In regards to leadership, a schedule also makes it clear when someone is slacking off. While we don’t use IM for constant conversation, it still provides a clear status of when someone arrives or leaves each day.

Coworking: Working alone, together

Unless you have the right office space at home, it can get really distracting. The question always comes up on whether an office away from home makes sense. My usual answer is that it depends on the person. Over the years, we have tried it all. The important thing is that each person has the flexibility to work where they are most productive and comfortable. With so many people working remotely these days, the concept of coworking has really caught on. I work at Independents Hall in Philadelphia, which provides a great balance between working at home and working in an office. As more of these spaces open, the rest of our team will be sure to jump on board.

Clear communication

With the right people and the right process, a solid communication process brings it all together. While there are plenty of tools to assist in communication, the real value is in setting communication guidelines. Nothing is more distracting than the Internet (I hate you Twitter). By focusing on communication tools first, it just feeds the fire. Here are some of our guidelines.

Daily meetings

The biggest disadvantage to a virtual team is not having the time to hang out. In the usual workplace, a lot of great ideas happen through daily chatter or over lunch. To emulate this environment, we setup very short meetings each morning in Campfire. It helps start the day, mess around, and catch up before we focus on work.

Kill the IM distration

Instant Messages do a great job of killing focus. Nothing is worse than tasks assigned over IM or someone distracting you in the middle of a thought. Over time, we learned how to minimize IM discussion. By not using IM, we are forced to use tools that encourage structured team discussion. We use Basecamp to post ideas or questions to the team, which can be answered at any time. We use Campfire to chat as a group, which keeps a record for future use. And we use FogBugz to assign and organize tasks and iterations. We’ve also used Twitter to notify each other about the status of a task or situation.

Get to know each other

It’s not always about business. Nothing helps a team gel more than learning about each others personal lives. It’s easier when you work in the same office, but in a virtual team you need to make time for it. We’ve celebrate with shots of vodka in Campfire. We send each other pictures of our home offices. We remember birthdays or occasions and announce them to the team. The personal discussions are always overlooked, so be sure to make the time.

It’s not all virtual. No matter what, you always need some face time. There’s nothing better than a team retreat to hang out with the team, celebrate success, and focus on strategy. Our last retreat was in Cyprus, and it was well worth it. When planning a retreat make sure that most of the time is casual. It’s a time to relax and get to know each other, rather than pushing business needs. The strategic conversations will happen naturally.

People, Process, Communication

Our experience working remotely is just one example. It’s important to adjust based on the needs and structure of your team. As I said, it all starts with the right people. With a great team that really strives for quality, the rest just happens. The concepts and process that Wildbit has created over the years has come from each person on the team, instead of a single manager or leader.

I’m very interested in hearing ideas or strategies that have worked for other teams. If you have any thoughts or questions, post a comment and I will reply.

Subversion is an impressive version control system that has been widely adopted. You’ve probably heard of it. You might have even tried to use it without success. Most people know they need it, but can’t get past the complexities of getting started. At Wildbit, we realized these problems and created Beanstalk, a hosted Subversion system designed to make version control painless. In order to understand Subversion, let’s start with the benefits and how it works.

Fear commitment? Drop that ball and chain.

With Subversion, you don’t have to worry about overwriting files or losing code. Subversion will keep a constant history of revisions for every file in your project. Each time changes are made to your code or files, they are preserved forever. Subversion allows you to view the changes to specific code, review with your team, and even roll back changes to previous versions. Think of it as a never-ending “undo” for the life of your projects.

In addition to revision history, Subversion improves team collaboration. Many people can update the same files without the risk of overwriting each others work. Subversion also provides a constant backup of your files in a central location, where you can share changes and updates with clients or team members. Since it preserves the history of your files, you can instantly compare multiple revisions of HTML mockups.

How does it work?

Subversion works as a client/server tool, similar to the way most designers use FTP. A Subversion server stores all of your files in a repository and a Subversion client is used to download (update) and upload (commit) file changes to the server. In the past, the process of setting up a server, finding a good Subversion client, and learning the basics of Subversion was a painful process. These days there are many tools and services available that allow almost anyone to get started. Let’s jump in.

Setting up a Subversion server

The first step is to setup the server. This used to be the most difficult step, but there are now various web applications that simplify this process. Services such as Beanstalk, Versionshelf, CVSDude, and RoundHaus make it easy to create a repository, manage users, and view activity. These services have literally taken the process from hours to minutes.

After signing up for one of the services, you can setup a repository. A repository is like a project, which defines the location where your files are stored. Each repository has a URL that is used to interact with the client and the server. As part of setting up the repository, you can either import an archive of existing files or start from scratch. An example repository URL would look something like http://svn.account.beanstalkapp.com/reponame where reponame is the name of your repository.

Installing the client

The next step is to setup a Subversion client on your computer. I feel like this is the part where most people give up. There are many options and most are poorly documented or difficult to install. Fortunately, there are a few installable clients that can make this process much easier without having to bother with the command line.

If you are on Windows, I highly recommend TortoiseSVN. This Subversion client seamlessly integrates into the Windows Explorer, making it easy to interact with your Subversion repositories. With a right-click, you are provided with the most common Subversion commands. The installation process is really straight-forward.

On Mac, you have several options. The biggest issue with Subversion on Mac is that you need to install Subversion as well as the GUI. Fortunately, there is a package from Martin Ott that will install Subversion for you. You can grab the file (My Subversion-1.4.4 Package) from the link and install it. After the install, you can choose a Subversion GUI client. I recommend either svnX or Textmate’s Subversion bundle. Details about the use of each product can be found on the respective web sites.

Basic Subversion usage

With a Subversion server, repository and client we are ready to start using Subversion for a project. The best way to explain Subversion is through a normal day to day work cycle. In this example, I will use Beanstalk (server) and svnX (client) to cover the common commands and terms.

svn checkout

The first step is to checkout the files from your subversion server. A checkout is the first download of files from the Subversion server to your computer. In your Subversion client you will insert the repository URL along with your username/password and perform a checkout. This will “checkout” all files from the repository to a directory on your computer. The screen shot below shows that we checked out several files. As you can see, I am using Subversion to edit this article.

working copy

The working copy is the local copy of files on your computer from your repository, which is created after your first checkout. Subversion will automatically keep track of changes in your working copy as long as you edit within the initial checkout directories. For now, I will edit this page in Textmate.

svn commit

After editing the file, we can commit (upload) the changes to the Subversion server. svnX will automatically find all new or changed files when you are ready to commit. Make sure to write a detailed comment that explains the changes.

changeset

Once the files are committed, Subversion will create a changeset that records the changes from the commit. Each changeset is assigned a revision number and is frozen in history in your repository. The changeset and revision is displayed in Beanstalk and is visible to other members on your team. The screen below shows that I modified the file subversion-for-desgners.html at Revision 2.

diff

When viewing a changeset or file in Beanstalk, you can see a “diff” of the changes. A diff is simply the difference between one revision and another, which will display the lines of code that were changed, added, or removed. This makes debugging and reviewing changes very easy, since you can review specific differences in each file between revisions. I went ahead and edited this line so we can see the “diff” between revision 2 and 3.

svn update

If you are working in a team it is important to have the latest version of files. An update is the command used to grab the latest version of files from the Subversion server. You should run this often to minimize any conflicts with team members.

svn log

The Subversion server will keep a constant log of everyone’s activity across your repositories. Most hosted Subversion services offer a section where you can view recent revisions and changesets. This is ideal if you need a snap shot of recent updates in your projects. The log will display the latest commits from each person as well as a direct link to the changeset.

much more

There are many other terms and commands, but the above are the most important to get started. Once you start using Subversion on a daily basis, you will learn how to resolve conflicts, revert files to previous revisions, and create a smarter workflow for your process. The best in-depth resource for Subversion is the book Version Control with Subversion. Instead of starting at the top, I recommend starting at the chapter Basic Work Cycle. This will help you discover the key advantages of Subversion without reading the entire book.

Taking it further

I tried to keep this as simple as possible, but once you get started with Subversion there are many more tools and benefits. Hosted Subversion services have some great tools that allow you to automatically deploy files from Subversion to your servers, notify your team when new changes are committed, integrate with your favorite web apps and allow anonymous access for open source projects. After you decide to use Subversion, you will wonder how you ever got any work done without it.

If you have any questions, feel free to post comments and I will reply quickly. You can also email us at support@beanstalkapp.com if you need help along the way. Our goal is to open up the world of version control for designers, so we are here to help.