Java game 2D overlapping shadows with Swing

Question

I am currently developing a 2D Java game using Swing as my primary drawing component. Every object has a shadow (BufferedImage) but every shadow overlaps other shadows. Is it possible to only have the shadows not overlap each other? Because I still want the shadows to draw over the player if the object is beneath it, and not if the object is above of the player. Here is a picture for clarity:

I have looked at alpha compositing, I guess I need Source Out? I also thought of having all the shadows (with no transparency) draw on one layer and then draw it with transparency but then it won't draw over the player and other objects like before.

I have a Draw object which is a JPanel and overrides the paintComponent method. Within this method I draw the floor of the current room and then I iterate over the list of objects that belongs to the current room and call each objects' draw method to draw everything.

The object draw method:

public void draw(Graphics g) {
    if (visible && checkInScreen()) {

        // The required drawing location
        int drawLocationX = getX() - globalCameraX;
        int drawLocationY = getY() - globalCameraY;

        if (shadow) {
            g.drawImage(shadowImages.get(imageIndex),
                    drawLocationX + shadowOffset.x + (getImageWidth()/2),
                    drawLocationY + shadowOffset.y, null);
        }
        g.drawImage(images.get(imageIndex), drawLocationX, drawLocationY, null);

        //Collisionbox
        if (SHOW_COLLISION_BOXES){
            g.setColor(Color.WHITE);
            g.drawRect(drawLocationX + getCollBoxX(), drawLocationY + getCollBoxY(), getCollBoxW() - getCollBoxX(), getCollBoxH() - getCollBoxY());
        }
    }
}

My apologies if this question has already been asked but I couldn't find something similar like this.

Answer 1

What I would do to solve this is to have a shadow-layer bitmap. By which I mean: have your shadow textures saved as a 2D array of boolean values (representing the position of a shadow pixel).

What you can do with this is to then logically or the shadow maps together to create a single layer, which can be layered behind the tree textures to create the shadows.

You may want to change the booleans to floats to represent the colour/intensity of the shadow, then have a larger calculation to merge the shadows together.

The below ShadowMap class is used to store the data for each shadow:

class ShadowMap {

    public int xPos, yPos;
    public boolean[][] array;

    public ShadowMap(int xPos, int yPos, boolean[][] array) {
        this.xPos = xPos;
        this.yPos = yPos;
        this.array = array;
    }

}

The ShadowLayer class creates a 2D array for the entire screen, containing if a shadow is present for each pixel:

class ShadowLayer {

    public static boolean[][] array = new boolean[SCREEN_WIDTH][SCREEN_HEIGHT];

    public static makeNew(ShadowMap[] shadows) {
        for (int x = 0; x < SCREEN_WIDTH; x++) {
            for (int y = 0; y < SCREEN_HEIGHT; y++) {
                array[x][y] = false;
            }
        }
        for (ShadowMap map : shadows) {
            for (int i = 0; i < SCREEN_WIDTH; i++) {
                for (int j = 0; j < SCREEN_HEIGHT; j++) {
                    // Logical or such that the pixel at (x, y) has a shadow 
                    // if any shadow map also has a shadow at pixel (x, y)
                    array[i + map.xPos][j + map.yPos] |= map.array[i][j];
                }
            }
        }
    }

}

Using this ShadowLayer class, you just need to darken each pixel on the screen if the ShadowMap has a shadow on the same pixel:

public static Color ajustPixelForShadows(int x, int y, Color pixel) {
    return ShadowMap.array[x][y] ? pixel.darken() : pixel;
}

Answer 2

I admit I'm not familiar with Swing so I'm not sure it is possible with that specific interface but the below solution could be used in a variety of 2D graphics engines.

You'll need an off-screen "shadow layer" to draw to that matches the screen dimensions. Initialize the shadow layer to being pure white.

For each object you draw from back to front (y-sorting), do the following, in order, with the shadow layer:

• Draw the object's shadow shape in a single solid dark grey color to the shadow layer

• Draw the object itself to the shadow layer as a pure white sprite (i.e. all non-transparent pixels in the object's bitmap are white)

Of course, also draw the object itself to the screen.

Then, once all objects have been drawn to both the screen and the shadow layer, draw the shadow layer to the screen using multiply blending. The multiply blend guarantees shadows will darken whatever they are drawn over (unlike alpha blend which, with very light shadows, could potentially actually lighten the colors they are drawn over). It will also make the pure white portions of the layer do nothing, which is what you want.

The above steps mean that after each object draws a shadow, it erases any shadows that would be underneath it in the final scene when it draws itself in white to the shadow layer. Therefore it won't cast a shadow on itself, and objects won't cast shadows over other objects that are technically in front of them.

Objects will still cast shadows onto other objects that are behind them as you wanted, since any parts of the shadow that haven't been erased by an overlapping object will still apply (or if they are erased, will be potentially re-drawn by a later object). And, since you are drawing the shadows as a single non-translucent color to the shadow layer, multiple shadows overlapping won't affect each other either, which was of course the main point.

You could modify this technique depending on what you have available. For example, instead of white you could use a fully transparent shadow layer initially and an "erase" blend mode [(src * 0) + (dst * (1 - srcAlpha))] to draw the objects that erase shadows underneath them. You could then use alpha instead of multiply blend if you prefer for drawing the shadow layer to the screen.