Bitmap Masks - Natural Selection

[Painter Essentials 3]
[Colour Management]
[OnOne Suite]
[Photo to Art]
[Photoshop Elements 4]
[Paint Shop Pro X]
[PhotoImpact 11]
[Photoshop CS2]


[Apps Overview]

[Camtasia Studio]
[CompuPic Pro 6]
[Digital Image Suite]
[Deep Paint 2]
[FotoFusion 3]
[Paint Shop Pro X]
[Painter IX]
[Painter Essentials 3]
[Photo Album 6]
[PhotoImpact 11]
[PhotoObjects 2]
[Photoshop CS2]
[Photoshop Elements 4]
[PhotoSuite 5]
[Piranesi 4]
[SnagIt 6]
[Texture Maker 2]


[Plugins Overview]

[AutoEye 2]
[DreamSuite 1]
[Eye Candy 4000]
[Genuine Fractals 3.5]
[Image Doctor]
[KnockOut 2]
[KPT 7]
[Mystical Lighting]
[Mystical Tint]
[OnOne Suite]
[pxl SmartScale]
[Virtual Painter]
[Xenofex 2]

[Articles & Tutorials

[Colour Management]
[Photo to Art]
[Adobe Macromedia]
[Extending Bitmaps]
[Image Modes]
[Elements Tutorial]
[App Roundup]
[Photoshop Patterns]
[Texture Library]
[Creative Roundup]
[Photo Bargains]
[Creative Suite]
[Tablet PC]
[Pen Input]
[Filter Tips]
[Photoshop Filters]
[Image Resampling]
[Image Resolution]
[Digital Video]
[Image Channels]
[Photo Apps Roundup]
[Image Management]
[Understand Halftones]
[Photoshop Vectors]
[Image Enhancement]
[Image Hoses]
[Producing Panoramas]
[DTP Compositing]
[Brush Control]
[Digital Cameras]
[Stock Photography]
[Bitmap Editability]
[Photoshop 5 Colour]
[Image Management 2]
[Layer Handling]
[3rd Party Solutions?]

[Home / What's New]
[DTP / Publishing]
[Vector Drawing]
[Bitmap / Photo Editing]
[Web Design]
[3D Software]
[All Reviews]
[All Articles / Tutorials]
[Book Search / Shop]
[Site Map / Search]

you can help support the site with a direct donation or by shopping via the following links:


Thank you!



Getting to grips with photo-editing selection

Tom Arah takes a look at the range of bitmap selection tools available and tells you how to make the most of them.

Selection by colour range

One of the major differences between people and computers is that our senses automatically make meaning of the data we are dealing with. Whenever we look at an image we immediately and automatically see what it consists of whether it is a mountain landscape, a wedding photo or whatever. To the computer, however, this level of meaning is completely absent. To the computer any image is simply a matrix of colour values and whether those pixels describe a sunflower or a skyscraper is unknown and irrelevant. The problem is that the professional photo-editor regularly has to work with this higher level of meaning when colour correcting or photo-compositing. As such, the ability to select and isolate areas of an image to be handled separately is an integral part of successful photo-editing.

The control over the selection process is crucial, but the software developers never seem to have appreciated its full importance so that the various methods of selection have been allowed to develop haphazardly over time. Moreover, when the developers are unsure of what they're doing it's hardly surprising that users are too. This month I hope to put that right by looking at the three generations of selection tools and how to make the most of each of them by understanding their different strengths and weaknesses. I'll be focusing on Photoshop, but all photo-editors have similar tools that work in similar ways.

First Generation Tools - Shape

The first generation selection tools are those that rely on the user to create the selection based on shape. The most basic shape of all is the rectangle and Photoshop offers its Marquee tool (shortcut "m") to make rectangular selections by dragging onscreen. Double-clicking on the tool opens up the Options palette where it's possible to set a fixed size or aspect ratio and to change the shape of the marquee to an ellipse or a single row or column. The last option can be particularly useful if you have a scratch on your scanner as it allows you to quickly copy over the column of pixels next to it. Much more flexible for selecting irregular shapes is the Lasso tool (shortcut "l") which allows freehand selections to be made by dragging, or straight line selections to be made by holding down the Alt key and clicking.

Once the selection has been created it is shown on screen by a flashing black and white border normally referred to as "marching ants." If you are working on a complex mask this can be distracting in which case you can use the View menu's Hide Edges command (Ctrl + H) to toggle the marching ants on and off. Dragging within the selection allows the selection outline to be repositioned and, for fine control, it is possible to nudge it into place with the cursor keys. Unlike other programs, however, there are no options for resizing, rotating or skewing the outline although Photoshop 5 is set to address the issue.

When you have made a selection you'll find that you can only affect the pixels within its borders. In other words you are protecting or "masking" those areas of the image that are not selected. This masking effect is one of the major reasons for selection, but it is also a common cause of bafflement - especially if you've turned off the marching ants outline - when you suddenly find you cannot edit the image. Another potential problem comes when you've created a selection, but it's of the area that you don't want to work on. Some programs such as Painter allow you to change the masking mode so that you can swap which areas of the image are protected, but Photoshop instead offers the Select: Invert command (Shift + Ctrl + I) which swaps the areas which are selected and masked. To remove the selection outline completely many users try the Delete command only to find the actual pixel information disappearing. In fact this is a very useful shortcut for filling large areas with the current background colour (with Alt + Delete filling it with the foreground colour), but it's crucial to know how to drop the current selection. In Photoshop this essential command is again unforgivably hidden away in the Select menu's None option (shortcut Ctrl + D).

Fine-tuning Selections

The trouble with simple, shape-based selections, of course, is that in the real world standalone ovals and rectangles are the exception rather than the rule. It's obviously necessary for more flexibility to be built into the system to allow the original selection to be expanded or refined. Most modern programs now offer a property bar where it's possible to set the selection mode to allow new areas to be added to the selection or removed from it. In Photoshop you just have to learn the method which is to hold down the Shift or Alt key respectively when dragging onscreen. Make sure you don't hit the Ctrl key by mistake or you'll end up moving the selection or the selected pixels.

Using the add and subtract selection modes together with the lasso tool allows some semblance of interactive fine-tuning, but it's still a chore dealing with fiddly areas. For difficult jobs it makes much more sense to be able to paint the selection as this allows the shape and size of the brush to be changed and for the selection to be gradually built up. Some programs offer a selection brush tool to allow this, but Photoshop instead offers a separate editing mode. Again this is hidden away so that many users have never realised it is there. At the bottom of the tool palette under the colour boxes are two commands for editing in standard mode and in quick mask mode (shortcut "q"). In quick mask mode it's possible to paint on the screen with black to add to the current mask, or in white to subtract from it. As you paint the masked area is shown partially covered by a colour you set by double-clicking on the command. When you change back to standard mode those areas not covered by the mask are automatically selected.

Tonal Masks

Using quick mask mode with black and white solid brushes, it's relatively straightforward to build up a simple selection. However, it's important to realise that if you have a different colour or a soft brush selected, the effects can be entirely different as the mask mode allows grayscale masks to be applied. If a mask is created with a 50% gray brush, for example, if you then applied a filter it would only have 50% of the effect. When understood, this opens up a whole range of effects. It is possible to use an airbrush, for example, to ensure that a sharpening effect is strongest around the face and then trails off. Alternatively, by applying a gradient to the quick mask, it is possible to create some striking effects where posterisation or hue shift increases across an image.

This ability to alter the strength of a mask is the basis for two other important features of selections: feathering and anti-aliasing. Feathering allows a selection to be given a fringe of increasing transparency to give it a soft vignette-like edge. The feathering effect is specified as a simple radius of pixels, but in fact Photoshop uses a more sophisticated algorithm that affects far more pixels. Anti-aliasing works on the same principle but instead of producing a softening effect is used to produce a smoothing effect by partially filling edge pixels to remove any visible "jaggies". It's impossible to see the effect of feathering or anti-aliasing as the selection is made, but the effects are clearer if a selection is converted to a layer and repositioned, or particularly if it is copied and pasted into a new file. Feathering can be set either before selection with the Options palette or afterwards with the Select: Feather command, but it's crucial to ensure that anti-aliasing is on before making a selection.

The Magic Wand

I said in the introduction that the actual pixel values of the image are irrelevant to the elements of meaning in the image, but that's not necessarily true. The elements in an image often stand out exactly because of their colour, so that the yellow sunflower stands out against the blue sky, for example. The second generation of selection tools recognised this and allowed the user to tap in to the colour information in the image to effectively automate the selection process. All packages now offer a "magic wand" tool of some description all working in a similar way. When the user clicks on a pixel with the wand (shortcut "w"), the program compares its colour values to all adjacent pixels and, if they fall within the current tolerance setting, they are automatically selected. This tolerance is set on a property bar with most modern packages and through the Options palette in Photoshop.

In fact, although I said the magic wand tool works on colour, that's not quite true. Rather than based on hue, the selection is actually determined by the brightness levels within the image's different channels. This explains why the tolerance is set between 0 - which will only select the exact same pixel values - and 255 - which will select the entire image - and why if you select a shadow or a highlight you will actually be selecting a smaller range than if you select a midtone. To take an example, if you are working with a RGB image and click on a pixel with values R:160, G:50, and B:10 and a tolerance of 32 you will end up selecting all adjacent pixels in the range R:128-192, G:18-82 and B:0-42. If the image mode is CMYK or LAB, however, clicking on exactly the same pixel will produce a completely different selection. It makes the selection process pretty arbitrary, but it does have one practical advantage. By looking at the individual channels you can often find greater contrast so that to select the sky, for example, you could choose to work in the Blue channel.

Sadly the arbitrary nature of the magic wand's selection is made worse by Photoshop's inflexibility. It's possible to change the tolerance level, but infuriatingly this will only affect the next selection. By holding down the Shift or Alt key it's possible to add to or subtract from the selection, but again this is very hit or miss. Fortunately, there are some important alternative options which again Photoshop hides away under its Select menu. The Select:Modify:Expand command allows the selection to be extended by a set number of pixels while the Select:Grow command (no shortcut!) allows the tolerance level to be automatically increased by its current setting. Most powerful of all is the Select:Similar command which selects all similar pixel values throughout the image. It's almost inevitable with such crude controls that unwanted holes will appear in the selection but, with no dedicated Remove Holes command, these have to be manually taken care of with the lasso or quick mask mode.

Colour Range

Ultimately, the magic wand tool only comes into its own for simple selections of discrete colours, selecting the black text in an image, for example, or the solid colours in a scanned logo. Trying to select the flesh tones in an image, or all the leaves on a tree, is a nightmare. A more sophisticated alternative is found in the Colour Range command. This is a separate dialog that allows various pre-set colour values to be selected automatically or for the user to select and gradually build up the colour values they are interested in by using the eyedropper tools (Shift to add, Alt to subtract). The huge advantage is that it's possible to hone in on the desired selection by interactively controlling the Fuzziness slider which automatically changes the range of colours that are selected.

This is a big step forward, but it's important to realise that the fuzziness setting works in a very different way to the Magic Wand's tolerance setting. While both affect the range of colours selected, with the magic wand tool a pixel is either selected or it isn't, but with the colour range command the further away a pixel's values are away from the seed values the less it is selected. In other words the Colour Range command automatically creates grayscale masks. In fact this tonal range is a big strength for colour corrections such as changing the hues in a model's dress, but it means that the selections are useless for creating objects by copying to a layer. The option to create solid selections would be a huge advance as would the ability to anti-alias the results to avoid obvious edges.

Intelligent Selection

Generally speaking these are the selection tools, from the Marquee through to the Colour Range dialog, that most users have been working with for the last few years. A third generation of tools though are now appearing that allow the user and computer to pool their respective skills to work together. A typical example is Photoshop 5's new Magnetic Lasso which allows the human eye and the computer's number-crunching to work together. As such the user roughly follows the outline of the object they are interested in and the computer automatically looks for sharp areas of contrast that mark the object's edge and follows those. Double-clicking on the tool opens up the options palette to control the level of contrast/snap and how often fixed nodes are put down.

With objects that stand out from their background the results are excellent with faster selection and greater accuracy. However with objects that aren't so clearly defined, for example against a shadow, the benefits are much more debatable. The major problem seems to be that Photoshop spots a line of contrast that you aren't interested in and runs off putting down its own selection outline and fixed nodes. This isn't an irresolvable problem as pressing the Delete key deletes the last node and by holding down the Alt key it's possible to return to manual lasso mode, but it does mean that for complex selections the tool can cause as much trouble as it solves.

Fourth Generation?

The magnetic tools are a step in the right direction but there's clearly still a lot of room for improvement before computer and user are really working together in perfect harmony. I thought I might have come across this ideal, and the fourth generation of selection tools, when I came across MagicMask ( Reading the introductory blurb for the program was eye-opening: "The company's ImageGenetics technology - based on extensive research in the fields of molecular genetics, fractal geometry and complexity theory - automatically decodes the way that nature interweaves colour and form in an image, enabling you to manipulate each separately." The claimed end result is that "MagicMask brings you patented colour editing tools that see an image much the same as you do."

The program might sound more like a cult than a Photoshop plug-in, but in practice it proved much more mundane. In fact the program actually works in a very similar way to existing tools offering its own versions of the marquee, lasso, mask brush, magic wand, magnetic lasso and colour range eye-droppers. Closer inspection though, showed that the program does offer a number of advantages. To begin with its colour selection is actually based on hue rather than channel brightness. Secondly each of its tools tends to be slightly superior to its Photoshop equivalent, so that the magic wand tool, for example, can be dragged to automatically select a whole range of colours. Most important of all, flexibility is built into the system so that each step in the selection process can be undone or retrospectively fine-tuned to control its effect.

If it is a fourth generation selection tool, MagicMask is so not because of its ImageGenetics technology, but because it takes a dedicated and completely thought-through practical approach to the whole process. However it also made me think seriously about what a real biologically-inspired solution could offer. On reflection it's clear that the distinction I made between the eye and the computer in the introduction might be true now, but it isn't an absolute. After all a baby actually has to learn to see by developing the ability to perceive patterns in the colour information provided by the eye. By building similar pattern recognition linked in to a knowledge base there's no reason why the computer of the future shouldn't be able to automatically recognise meaningful objects, such as the people in an image, in the same way as current OCR packages can recognise text.

Maybe the idea of ImageGenetics and biologically-inspired natural selection has more going for it than it appeared. In the meantime, however, I'd seriously advise getting to grips with today's more artificial solutions.

Tom Arah


Hopefully you've found the information you were looking for. For further information please click here.

For free trials and special offers please click the following recommended links:

For further information on the following design applications and subjects please click on the links below:

[3D], [3ds max], [Adobe], [Acrobat], [Cinema 4D], [Corel], [CorelDRAW], [Creative Suite], [Digital Image], [Dreamweaver], [Director], [Fireworks], [Flash], [FreeHand], [FrameMaker], [FrontPage], [GoLive], [Graphic Design], [HTML/CSS], [Illustrator], [InDesign], [Macromedia], [Macromedia Studio], [Microsoft], [NetObjects Fusion], [PageMaker], [Paint Shop Pro], [Painter], [Photo Editing], [PhotoImpact], [Photoshop], [Photoshop Elements], [Publisher], [QuarkXPress], [Web Design]

To continue your search on the site and beyond please use the Google and Amazon search boxes below:

Web independent, informed, intelligent, incisive, in-depth...

All the work on the site (over 250 reviews, over 100 articles and tutorials) has been written by me, Tom Arah It's also me who maintains the site, answers your emails etc. The site is very popular and from your feedback I know it's a useful resource - but it takes a lot to keep it up.

You can help keep the site running, independent and free by Bookmarking the site (if you don't you might never find it again), telling others about it and by coming back (new content is added every month). Even better you can make a donation eg $5 the typical cost of just one issue of a print magazine or buy anything via or (now or next time you feel like shopping) using these links or the shop - it's a great way of quickly finding the best buys, it costs you nothing and I gain a small but much-appreciated commission.

Thanks very much, Tom Arah

[DTP/Publishing] [Vector Drawing] [Bitmap/Photo] [Web] [3D]
[Reviews/Archive] [Shop]  [Home/What's New]

Copyright 1995-2005, Tom Arah, Please get in contact to let me know what you think about a particular piece or the site in general.