Reading a cubemap pixel by direction in Unreal Engine 5

2023-06-29 #unreal

In order to implement an early version of pre-rendered 2D skybox I needed to compare the skyboxes with sun enabled and disabled in order to obtain the required tinting of the sun for 2D skybox (so the sun doesn’t render on top of it, shining right through dense clouds).

This system is no longer used, now a texture mask is used instead to remove stars, sun and moon and other background objects from being rendered where the clouds are.

Here is the function for sampling the cube map:

FLinearColor USynesthesiaBlueprintFunctions::SampleTextureCubeByDirection(UTextureRenderTargetCube* RenderTarget, const FVector& Direction)
{
	if (!RenderTarget) return FLinearColor::Black;

	// Get the correct resource
	FTextureRenderTargetResource* RenderTargetResource = RenderTarget->GameThread_GetRenderTargetResource();
	if (!RenderTargetResource) return FLinearColor::Black;
	FTextureRenderTargetCubeResource* CubeResource = RenderTargetResource->GetTextureRenderTargetCubeResource();
	if (!CubeResource) return FLinearColor::Black;

	// Normalize the direction vector
	const FVector NormalizedDirection = Direction.GetSafeNormal();

	// Get the correct cube face for the normalized direction
	ECubeFace CubeFace = ECubeFace::CubeFace_NegX;
	const double MaxAbsValue = FMath::Max3(FMath::Abs(NormalizedDirection.X), FMath::Abs(NormalizedDirection.Y), FMath::Abs(NormalizedDirection.Z));
	if (FMath::Abs(NormalizedDirection.X) == MaxAbsValue) {
		CubeFace = NormalizedDirection.X > 0 ? ECubeFace::CubeFace_PosX : ECubeFace::CubeFace_NegX;

	} else if (FMath::Abs(NormalizedDirection.Y) == MaxAbsValue) {
		CubeFace = NormalizedDirection.Y > 0 ? ECubeFace::CubeFace_PosY : ECubeFace::CubeFace_NegY;

	} else if (FMath::Abs(NormalizedDirection.Z) == MaxAbsValue) {
		CubeFace = NormalizedDirection.Z > 0 ? ECubeFace::CubeFace_PosZ : ECubeFace::CubeFace_NegZ;
	}

	// Calculate coordinates of the point on the selected cube maps cube face
	float CubeFaceU = 0.0f;
	float CubeFaceV = 0.0f;
	const FVector FaceDirection = NormalizedDirection.GetAbs();
	if (CubeFace == ECubeFace::CubeFace_PosX) {
		CubeFaceU = -NormalizedDirection.Z / FaceDirection.X;
		CubeFaceV = -NormalizedDirection.Y / FaceDirection.X;

	} else if (CubeFace == ECubeFace::CubeFace_NegX) {
		CubeFaceU = NormalizedDirection.Z / FaceDirection.X;
		CubeFaceV = -NormalizedDirection.Y / FaceDirection.X;

	} else if (CubeFace == ECubeFace::CubeFace_PosY) {
		CubeFaceU = NormalizedDirection.X / FaceDirection.Y;
		CubeFaceV = NormalizedDirection.Z / FaceDirection.Y;

	} else if (CubeFace == ECubeFace::CubeFace_NegY) {
		CubeFaceU = NormalizedDirection.X / FaceDirection.Y;
		CubeFaceV = -NormalizedDirection.Z / FaceDirection.Y;

	} else if (CubeFace == ECubeFace::CubeFace_PosZ) {
		CubeFaceU = NormalizedDirection.X / FaceDirection.Z;
		CubeFaceV = -NormalizedDirection.Y / FaceDirection.Z;

	} else if (CubeFace == ECubeFace::CubeFace_NegZ) {
		CubeFaceU = -NormalizedDirection.X / FaceDirection.Z;
		CubeFaceV = -NormalizedDirection.Y / FaceDirection.Z;
	}

	// Read correct pixel from the resource
	TArray<FFloat16Color> ImageData;
	if (!CubeResource->ReadPixels(ImageData, FReadSurfaceDataFlags(ERangeCompressionMode::RCM_UNorm, CubeFace))) return FLinearColor::Black;

	// Calculate pixel coordinates of the point on selected cubemap face
	const int32 SizeX = CubeResource->GetSizeX();
	const int32 SizeY = CubeResource->GetSizeY();
	const int32 U = FMath::RoundToInt((CubeFaceU + 1.0f) * 0.5f * (SizeX - 1));
	const int32 V = FMath::RoundToInt((CubeFaceV + 1.0f) * 0.5f * (SizeY - 1));
	const int32 Offset = V * SizeX + U;

	// Sample the cubemap data
	if (ImageData.IsValidIndex(Offset)) {
		return ImageData[Offset].GetFloats();
	} else {
		return FLinearColor::Black;
	}
}

And here is the function for saving a cube map to disk, just in case it might be useful in a similar context:

UTextureCube* USynesthesiaBlueprintFunctions::RenderTargetCreateStaticTextureCubeEditorOnly(UTextureRenderTargetCube* RenderTarget, FString InName, TextureCompressionSettings CompressionSettings, TextureMipGenSettings MipSettings)
{
#if WITH_EDITOR
	// Save the render target image as an asset
	if (!RenderTarget) { // Invalid RT
		FMessageLog("Blueprint").Warning(LOCTEXT("RenderTargetCreateStaticTextureCube_InvalidRenderTarget", "RenderTargetCreateStaticTextureCubeEditorOnly: RenderTarget must be non-null."));
		return nullptr;

	} else if (!RenderTarget->GetResource()) { // Invalid RT resource
		FMessageLog("Blueprint").Warning(LOCTEXT("RenderTargetCreateStaticTextureCube_ReleasedRenderTarget", "RenderTargetCreateStaticTextureCubeEditorOnly: RenderTarget has been released."));
		return nullptr;

	} else { // Valid inputs, generate static texture
		FString Name;
		FString PackageName;
		IAssetTools& AssetTools = FModuleManager::Get().LoadModuleChecked<FAssetToolsModule>("AssetTools").Get();

		// Use asset name only if directories are specified, otherwise full path
		if (!InName.Contains(TEXT("/"))) {
			const FString AssetName = RenderTarget->GetOutermost()->GetName();
			const FString SanitizedBasePackageName = UPackageTools::SanitizePackageName(AssetName);
			const FString PackagePath = FPackageName::GetLongPackagePath(SanitizedBasePackageName) + TEXT("/");
			//AssetTools.CreateUniqueAssetName(PackagePath, InName, PackageName, Name);
			PackageName = PackagePath + InName;
		} else {
			InName.RemoveFromStart(TEXT("/"));
			InName.RemoveFromStart(TEXT("Content/"));
			InName.StartsWith(TEXT("Game/")) == true ? InName.InsertAt(0, TEXT("/")) : InName.InsertAt(0, TEXT("/Game/"));
			//AssetTools.CreateUniqueAssetName(InName, TEXT(""), PackageName, Name);
			PackageName = InName;
		}
		const int32 LastSlashIndex = InName.Find(TEXT("/"), ESearchCase::CaseSensitive, ESearchDir::FromEnd);
	    if (LastSlashIndex != INDEX_NONE && LastSlashIndex < InName.Len() - 1) {
	        Name = InName.RightChop(LastSlashIndex + 1);
	    } else {
	        Name = InName;
	    }

		// Create the package
		UPackage* Package = CreatePackage(*PackageName);
		if (!Package) {
			UE_LOG(LogSynth, Error, TEXT("Failed to create package %s"), *PackageName);
			return nullptr;
		}
		Package->FullyLoad();

		// Create or overwrite the texture
		UTextureCube* NewTexture = Cast<UTextureCube>(RenderTarget->ConstructTextureCube(Package, Name, RenderTarget->GetMaskedFlags() | RF_Public | RF_Standalone));

		// If the texture is valid, set its parameters and return it
		if (NewTexture != nullptr) {
			// Package needs saving
			//NewTexture->MarkPackageDirty();
			Package->SetDirtyFlag(true);
			Package->PackageMarkedDirtyEvent.Broadcast(Package, true);

			// Notify the asset registry
			FAssetRegistryModule::AssetCreated(NewTexture);

			// Update Compression and Mip settings
			NewTexture->CompressionSettings = CompressionSettings;
			NewTexture->MipGenSettings = MipSettings;
			NewTexture->PostEditChange();
			return NewTexture;
		}
		FMessageLog("Blueprint").Warning(LOCTEXT("RenderTargetCreateStaticTextureCube_FailedToCreateTexture", "RenderTargetCreateStaticTextureCubeEditorOnly: Failed to create a new texture."));
	}
#else
	FMessageLog("Blueprint").Error(LOCTEXT("RenderTargetCreateStaticTextureCube_RuntimeFailedToCreateTexture", "RenderTargetCreateStaticTextureCubeEditorOnly: Can't create TextureCube at run time. "));
#endif
	return nullptr;
}

Synesthesia: Re-entry effects improvements

2023-06-28 #synesthesia

I have improved the sprite for the re-entry effect. It still needs some polishing, but now the re-entry effect looks like this up-close.

An extra bonus picture:

Plus a video of the sky, sun, moon and clouds all blending together with the mask:

Synesthesia: Starfield improvements

2023-06-27 #synesthesia

Tachy, a friend of mine, helped me considerably by calculating a star catalog for the year 2994 (when the events of the game take place). Some of the stars experience a considerable drift over the 1000 years - it might not be a very important detail to most players, however it is a nice little detail to have.

Here is a simple comparison between the two starfields:

The newer year 2994 catalog also contains way more stars - 119,614 total. The older catalog has been truncated to only magnitudes brighter than 9.99, so it only listed 28,593 stars.

The synths sensors are sufficiently sensitive to pick out faint stars, so in the end I plan to include all ~120,000 stars (though possibly less based on performance settings, however it does not seem to be considerable for the particle system).

Below are the two starfields in higher resolution. These images can be opened at a higher resolution, if you want to take a closer look.

Epoch 2000

Epoch 2994:

Here is the video of the star field movement over time and how it blends together with the clouds using the mask shown in an earlier post.

Synesthesia: Debris re-entry effects

2023-06-26 #synesthesia

While working on the sky system, I had a thought about adding random space debris re-entering atmosphere. It would be a really telling and interesting piece of narrative that illustrates the state of the world - in the setting of Synesthesia, in year 2994, most of the space operations have been abandoned and overall there is no organized effort to monitor space debris anymore.

There are still millions of objects in orbit - many of these are private satellites, spacecraft and other assorted hardware. There are a few large objects (space stations and old spacecraft) which will eventually decay and re-enter atmosphere. All sorts of debris routinely enters the Earth atmosphere.

A large proportion of this debris is the result of the progressing Kessler syndrome, making any space travel dangerous and forcing people to abandon many of the space-based operations.

The player would be able to see these happen randomly. Many of them may be too small to notice unless they get lucky, however there may be a really big event once in a while.

The re-entry effect is implemented as a particle system combined with a fairly simple thermal simulation. The debris objects are spawned at the height of about 120-140 km and are given a random, statistically distributed direction & entry angle.

The objects continue to fly free and a simple break-up model is used to calculate when the object finally burns up. It is a fairly simplistic model, it is possible that I might improve it in the future.

One definite improvement I want to make is using better sprites - it would make the re-entry trails much more distinct, vivid, and less blurry. It would also emphasize multiple objects traveling in the re-entry path as the main object continues to shed debris.

The video below shows a better look at the hot entry and break up of the object. The re-entry event, if detected by players information systems, will also be recorded in the minor event log.

The re-entry events are defined by the following parameters:

Object mass - this defines how much debris is generated and how long can the object withstand high temperature.
Burn rate - this defines how fast temperature burns up the mass of the object or causes debris shedding events. The effective burn rate is proportional to this value and the current temperature.
Entry vector - the simulation has a simple exponential model of atmosphere, so shallow and steep re-entries look distinctly different.

A combination of these parameters can be used to set up a re-entry that looks most artistically appealing. The sky system creates random procedural re-entry events. For special scripted events, the same particle system can be created manually and multiple systems can be combined to create an appearance of a very large object breaking up.

Here is a close-up picture of the re-entry event. For prototyping, I’ve been using a simple spherical sprite, which makes the entire trail look quite blurry. The quality should improve considerably when the sprite will be more drop-shaped, with a large glowing core and a small trail following it.

Re-entry trail. Something is burning up out there

In addition to immersion, I want to include the re-entry effects in at least two elements of the narrative itself:

A side quest, during which the player can force a spacecraft in orbit to burn its engines for re-entry. While not having effect on the main quest, it would be narratively connected to the plot events.
- The destruction of this spacecraft would be done to hamper the mercenaries and Eden, the primary antagonists of the game.
- Most likely, this spacecraft will be an orbital weapon station, somewhat resembling SOL from Akira
A random event, during which an object re-enters over the EDZ-01 and visibly reaches surface.
- After re-entry, a glowing object falls somewhere in an obvious area, allowing player to go investigate and find this object.
- Possibly, the object is a new friend or an enemy, as well as some unique hardware.
- The side quest rewards player for the exploration they had to do in order to find the crashed object.

Synesthesia: Gameplay systems and elements

2023-06-25 #synesthesia

Primary Gameplay Loop

The primary loop is explore - cover - shoot. The player moves around complicated urban terrain, engaging human and creature enemies in tactical gunfights, managing their resources (maintaining weapons, keeping up with ammo counts, making sure equipment matches the enemies player is facing).

The game is influenced by S.T.A.L.K.E.R. series, Fallout: New Vegas and Arma 3. The primary activities in this game are:

Searching for technical artifacts
Interacting with NPC characters
Engaging in gunfights
Exploring the environment
Doing special narrative quests
Managing ammunition and supplies, weapon condition

→ Synesthesia Gameplay Systems and Elements

Page 1 of 2

Older posts →