Layout Compositing - The Successful Edge

[Painter Essentials 3]
[Colour Management]
[OnOne Suite]
[Photo to Art]
[Photoshop Elements 4]
[Paint Shop Pro X]
[PhotoImpact 11]
[Photoshop CS2]


[Apps Overview]

[Camtasia Studio]
[CompuPic Pro 6]
[Digital Image Suite]
[Deep Paint 2]
[FotoFusion 3]
[Paint Shop Pro X]
[Painter IX]
[Painter Essentials 3]
[Photo Album 6]
[PhotoImpact 11]
[PhotoObjects 2]
[Photoshop CS2]
[Photoshop Elements 4]
[PhotoSuite 5]
[Piranesi 4]
[SnagIt 6]
[Texture Maker 2]


[Plugins Overview]

[AutoEye 2]
[DreamSuite 1]
[Eye Candy 4000]
[Genuine Fractals 3.5]
[Image Doctor]
[KnockOut 2]
[KPT 7]
[Mystical Lighting]
[Mystical Tint]
[OnOne Suite]
[pxl SmartScale]
[Virtual Painter]
[Xenofex 2]

[Articles & Tutorials

[Colour Management]
[Photo to Art]
[Adobe Macromedia]
[Extending Bitmaps]
[Image Modes]
[Elements Tutorial]
[App Roundup]
[Photoshop Patterns]
[Texture Library]
[Creative Roundup]
[Photo Bargains]
[Creative Suite]
[Tablet PC]
[Pen Input]
[Filter Tips]
[Photoshop Filters]
[Image Resampling]
[Image Resolution]
[Digital Video]
[Image Channels]
[Photo Apps Roundup]
[Image Management]
[Understand Halftones]
[Photoshop Vectors]
[Image Enhancement]
[Image Hoses]
[Producing Panoramas]
[DTP Compositing]
[Brush Control]
[Digital Cameras]
[Stock Photography]
[Bitmap Editability]
[Photoshop 5 Colour]
[Image Management 2]
[Layer Handling]
[3rd Party Solutions?]

[Home / What's New]
[DTP / Publishing]
[Vector Drawing]
[Bitmap / Photo Editing]
[Web Design]
[3D Software]
[All Reviews]
[All Articles / Tutorials]
[Book Search / Shop]
[Site Map / Search]

you can help support the site with a direct donation or by shopping via the following links:


Thank you!



The problems and potential of layout compositing

Tom Arah shows you the secrets of successful compositing in all your publishing work.

Compositing your layouts 

Bitmap Compositing

A while back (RW43) I looked at the massive potential but inherent complexities of vector-based transparency. At the time I said that handling transparency in a bitmap-based context was comparatively simple and that's certainly true when working within your bitmap editor. The almost universal solution, pioneered by Micrografx Picture Publisher and then exploited by Adobe Photoshop, is the use of layers. Essentially these act like a stack of transparent acetate sheets so that wherever a pixel in the top layer is transparent, the topmost underlying pixel shows through. This means that to create a photomontage all you have to do is make a selection, copy it to its own layer and then position the layer where you want it over the image background.

More importantly, in addition to this basic on-off transparency, layers can provide gradations of transparency. In the basic photomontage described above, for example, the object on the layer would still look clearly separate from the underlying image as if it had been crudely cut and pasted. To make the composition seem more organic when you select the object simply ensure that anti-aliasing is turned on. This varies the transparency of the selection's edge pixels so that the colours in the object's edge and the background seem to blend slightly to produce near seamless results. Larger scale compositing effects, such as uniformly varying the opacity of the layer or interactively painting transparency onto a layer mask, can also be created thanks to variable transparency.

All these advanced semi-transparency effects are possible because each layer pixel is defined not just by its 24-bit RGB colour values but by an 8-bit transparency level. Thanks to this transparency information the compositing capabilities of bitmap editors are immense and enable any layer-based editor right down to Paint Shop Pro to produce stunningly realistic - or surrealistic - photomontages. However, there's still a big initial stumbling block which prevents most users exploiting such capabilities and that's the complexities involved in selecting and isolating the object in the first place (see RW46). As such I was intrigued when a number of readers got in touch after my piece on royalty-free clipart to suggest I take a look at Hemera's Photo-Objects (50 inc VAT, MediaGold 0171 372 9733).


Photo-Objects is a budget two CD collection of 10,000 bitmap objects where all the hard-work of selecting the objects and creating smooth anti-aliased semi-transparent edges has been done for you. The fact that the photo-realistic images are not rectangular immediately makes them eye-catching and attention-grabbing or, as Hemera's slogan puts it, "gives your images an edge". More than this, the fact that the images have irregular shapes gives them another advantage when it comes to producing layouts. Rather than the commonplace rectangular grids you see everyday, the shape of an image can demand more interesting treatment with text runarounds producing a more integrated layout with a strong internal logic. In other words, the use of image objects can help make your designs seem more unusual yet more natural at the same time.

Transparency Problems

Of course to achieve this the Photo-Objects images have to be incorporated into your workflow and it's here that problems begin to appear. To begin with, the typical 600 x 600 pixel resolution of the images limits their output size to a few inches square before pixelization will become noticeable. Far more important, and not just relevant to Photo-Objects images but to all bitmaps with transparency, is the question of how you can access and use the image while keeping its transparency intact. Initially this doesn't look like it should be an issue as Photo-Objects allows you to simply drag and drop images directly into all your applications. That's all very well but immediately we hit serious trouble - the transparent background is automatically and infuriatingly converted to solid white!

For use in a program like Word there's nothing we can do about this. If text runaround is switched on, our bitmap image acts like what it is, a typical unexciting rectangular bitmap; if it's switched off, the text behind is entirely obscured. With a DTP program like PageMaker, however, there is at least a partial workaround. By sending the image behind the text blocks and then setting up an irregular text runaround the problem seems to be solved. The image transparency is still lost but, as the white pixels are no longer obscuring the text, the desired effect is restored.

Or rather it is so long as your design has a white background. If you want to place your image object on a coloured background, however, you're back to square one as the white pixels obscure everything below them and completely destroy your carefully thought-out design. If you are trying to drop your image object into a tinted box, for example, you are in serious trouble. From the mail I get it's clear that this is an incredibly common problem and everyone who comes across it assumes that they must have done something wrong. Sadly they haven't.


There isn't an instant solution to the problem, but again there is a workaround: change your image's background to match your layout's. Photo-Objects makes this simple by allowing you to change your image's background colour with the Windows colour selector or with an eyedropper tool that can pick up the desired colour from anywhere onscreen. If you're creating a presentation in PowerPoint, for example, you can copy your slide background to the Photo-Objects image and it will automatically convert the anti-aliased edge transparency into a smooth colour-based blend.

This mimics the full 8-bit transparency effect and is a big advance but it's by no means ideal. The image must again be sent behind all other objects if it's not to obscure them, for example, and if you change the slide's background colour your object will be left surrounded with a disconcerting halo based on the former colour. More importantly if your background isn't a uniform flat colour then the workaround falls flat on its face.

GIF Web Transparency

Surely there has to be a better solution, and the obvious question is why can't you just store the transparency information within the image file? In fact you can store transparency in one image format: the humble GIF. Photo-Objects offers saving to transparent GIF as a way of accessing its images but a bug seemed to prevent this from working as intended. By loading the GIF into Photoshop and using the Export GIF89a dialog, however, it's simple enough to set up. GIF is an indexed format so all you have to do is specify which of the colours in the image's palette you want to be treated as transparent which in this case simply means clicking on the single background colour.

It's a simple procedure and an important effect that allows web pages to be created where the GIF images seem an integral part of the layout rather than semi-detached afterthoughts. When you look closely at the edge of the GIF, however, you can soon see the format's major limitation. The indexed nature of the GIF format can only offer the crude on/off transparency effect so that the smooth effect of true anti-aliased transparency can again only be mimicked by colour-mixing. Again this means that if the GIF background colour isn't exactly the same as the design background, a halo is produced.

Without any true interaction between the colours in the GIF and the colours below this isn't true transparency then, but it is a major step forward and the best solution for transparency handling for both web and general screen-based work - at least until PNG support becomes more common. Unfortunately the other limitations of GIF with its restriction to a maximum of 256 RGB-based colours immediately rule it out for serious design work. Clearly what we need is a way to store a full 8-bit transparency mask in a bitmap format suitable for professional design work.

What about high-end design?

Photo-Objects seems to offer just this with another of its export options, "Graphics Application that supports 32-bit image with transparency mask" which allows its images to be saved to TIFF. This sounds exactly what we're after but, when trying to load a test TIFF of a PhotoObject complete with drop shadow into PageMaker or Quark, the result is definitely not what we want with the image background again rendered as solid white and the subtle drop shadow rendered as solid black! Loading the image into Photoshop produces exactly the same results. Again we seem to be back to square one.

However, all is not lost. The full transparency information is there in the TIFF, but it is stored as an 8-bit alpha channel. To bring the transparency into play we have to use the channel as a selection mask by Ctrl-clicking on it in the Channels palette. We can then copy the selection to the clipboard, copy and paste it into a new layer and then delete the background layer. At last we have accessed the transparency effect in all its glory complete with smooth edges and subtle drop shadow.

The process can hardly be called self-evident, but we've finally got the Photo-Object complete with transparency into Photoshop where it can now be used to create sophisticated compositions. More to the point as Photoshop is the designers' choice of bitmap editor, surely we can get now get it out to use it in our DTP-based design work? Unfortunately it's not that simple. Select the Save command and, because our image contains transparency, the only option on offer is to save to Photoshop's native PSD format, a format that none of the major DTP applications supports directly.

Hope is at Hand

Clearly we need to export the image to a supported format but trying any of Photoshop's Save A Copy options immediately converts the solid transparent pixels to white and semi-transparent pixels to a combination with white. Square one again! By now many users are tearing out their hair, but hope is at hand. Photoshop 5 now offers a dedicated Export Transparent Image command (though for reasons best known to Adobe this has been hidden away under the Help menu). Surely this must be the answer.

In fact the command is a wizard that walks the user through the necessary steps of creating a transparent image either as a GIF as we've seen for online use or - hallelujah - as a TIFF or EPS for print use. Load the resultant image into PageMaker, Quark or Ventura and it looks as if the wizard has truly done magic as the imported bitmap is indeed no longer rectangular and so can be placed over any background. At last we have a way of producing DTP layouts with layered transparent images that can be used to produce commercial CMYK print.

Closer inspection, however, is once again depressing. For our trial image with drop shadow, for example, the shadow has simply been dropped. Zoom in on the image edge too and it's clear that there's no 8-bit anti-aliasing to smooth the transition. Even worse the edge is veru crude with clear lines apparent almost as if the object had been cut out with scissors and crudely stuck in place. The inaccuracy of the edges even means that odd patches of the background colour are included as well! Our image might finally have an edge but this is hardly cutting edge control.

The obvious questions are how is the Photoshop wizard producing its transparency effect and why is it so crude? The answers are revealed if you look at Photoshop's History palette to see the steps the wizard undertakes. This shows that the wizard works by converting the non-transparent image selection to a path and then uses the Path palette's Clipping Path command to save this into the file. This explains everything. The transparency effect in the DTP program isn't pixel-based at all, but is effectively being mimicked by creating a vector-defined object with a bitmap fill. No wonder all graduated transparency effects are out of the question and the object looks like it has been crudely cut and pasted.

Clipping Paths

Of course there has to be a reason why supposedly high-end publishing work should be reduced to such an awkward scissors-and-paste style workaround and the answer is simple: Postscript-compatibility. Postscript by its nature is stack-based with all objects in the stack obscuring those below so that it's impossible to set up subtle transparency effects where every pixel in the stack can have an input into the final output. In fact it was largely to avoid such memory and CPU-intensive pixel-based processing that Postscript was devised in the first place. In this one area then, Postscript's strengths become an insurmountable problem.

It looks like we're stuck with clipping paths then for producing pseudo-transparency for Postscript-based commercial print so clearly it's important to make the most of them. Most DTP packages can access the clipping path information stored in the image, but offer no further control. The one application providing serious support is QuarkXPress 4 which offers a dedicated Item>Clipping tab (Alt + Ctrl + T). Using this you can access any embedded path or alpha channel to use as the current clipping path. Alternatively you can generate your own clipping path based on the image's paths, channels or non-white areas and set how closely the path follows the image object. Best of all with the Item>Edit>Clipping Path (Ctrl + Shift + F10) command you can even edit the path directly.

Making the most of a bad job

The clipping control offered by Quark is a massive advance, but in many ways it's still only papering over the cracks. You can tidy up the transparency effect to remove any eye-catching patches of background colour, for example, but even then the object still looks unnatural and at best semi-detached from you layout. Surely there has to be a better way of controlling image transparency in your designs? What's needed is a middle way between the bitmap editor's pixel-based and the DTP program's Postscript-based handling.

This intermediate approach is exactly what's being offered by the more adventurous of the drawing packages, such as Corel Xara and Corel Draw both of which allow their own objects and imported bitmaps to be given graduated transparency effects. The undoubted leader of the pack, however, is Deneba's recently released Canvas 6. The compositional control this offers is little short of amazing, acting like a cross between a dedicated drawing, photo-editing and DTP program. Shapes, photos and text can all be integrated within your layout with advanced control offered over each.

Real compositing power

What really sets Canvas 6 apart, however, is its SpriteLayer technology which acts as a dedicated and total solution to all the problems we've encountered above. Using the program's bitmap selection tools, for example, an image object like a butterfly can be isolated from an imported photo. It can then be cut and pasted repeatedly into a layout complete with true anti-aliased seamless edges - you can forget about clumsy clipping paths. More than this, by changing the object's transfer mode to darken or invert, for example, the butterfly can be made to interact creatively with all objects below. Best of all, objects including imported photos or blocks of text can be treated just as they are within a layer-based bitmap program so that you can change opacity, apply gradient fades or even interactively paint on subtle transparency effects.

Postscript v Pre-Ripped

The end result is seamless and organic compositions which look as if they must have taken months to devise let alone create, but which remain completely and instantly editable. This is a different world to the clumsy compositing workarounds within traditional DTP packages so what's the downside? The problem of course is Postscript. As we've seen Postscript doesn't natively support such advanced effects so the program must produce them itself. What this means is that Canvas must pre-RIP (raster image process) the design, outputting any SpriteLayer objects and any objects they overlap as bitmaps.

Such rasterization is clearly less than ideal in terms of efficiency and therefore cost and also raises the spectre of the designer's nightmare - colour-shifting. It's also not well-suited for the publishing industry's increasing moves towards PDF as a distribution and archiving medium. On the other hand, once you've seen the creative potential of Canvas 6 the thought of being permanently stuck in a flat Illustrator-style world is painful to contemplate. So does this herald a split into two opposing design camps - Postscript versus pre-RIPped?

Not necessarily. At the recent Seybold conference, Adobe's John Warnock and Chuck Geschke revealed that adding true transparency handling to Postscript and PDF was a top priority with a dedicated lab working full time on the problem. Assuming that Adobe does deliver the capability - and don't ask me how - then finally high-end design will be able to move onto a new level offering Photoshop-style compositing control within a truly integrated DTP-drawing-bitmap environment.

Until that happy day arrives, however, I'm afraid it's well worth getting to grips with the workarounds.

Tom Arah

March 1999

Hopefully you've found the information you were looking for. For further information please click here.

For free trials and special offers please click the following recommended links:

For further information on the following design applications and subjects please click on the links below:

[3D], [3ds max], [Adobe], [Acrobat], [Cinema 4D], [Corel], [CorelDRAW], [Creative Suite], [Digital Image], [Dreamweaver], [Director], [Fireworks], [Flash], [FreeHand], [FrameMaker], [FrontPage], [GoLive], [Graphic Design], [HTML/CSS], [Illustrator], [InDesign], [Macromedia], [Macromedia Studio], [Microsoft], [NetObjects Fusion], [PageMaker], [Paint Shop Pro], [Painter], [Photo Editing], [PhotoImpact], [Photoshop], [Photoshop Elements], [Publisher], [QuarkXPress], [Web Design]

To continue your search on the site and beyond please use the Google and Amazon search boxes below:

Web independent, informed, intelligent, incisive, in-depth...

All the work on the site (over 250 reviews, over 100 articles and tutorials) has been written by me, Tom Arah It's also me who maintains the site, answers your emails etc. The site is very popular and from your feedback I know it's a useful resource - but it takes a lot to keep it up.

You can help keep the site running, independent and free by Bookmarking the site (if you don't you might never find it again), telling others about it and by coming back (new content is added every month). Even better you can make a donation eg $5 the typical cost of just one issue of a print magazine or buy anything via or (now or next time you feel like shopping) using these links or the shop - it's a great way of quickly finding the best buys, it costs you nothing and I gain a small but much-appreciated commission.

Thanks very much, Tom Arah

[DTP/Publishing] [Vector Drawing] [Bitmap/Photo] [Web] [3D]
[Reviews/Archive] [Shop]  [Home/What's New]

Copyright 1995-2005, Tom Arah, Please get in contact to let me know what you think about a particular piece or the site in general.