Get color of sprite from ray pickingInfo

Censor · October 6, 2022, 10:49am

Is it possible to get the color of the sprite at the intersection with a given ray?

This topic is about the same thing but for textures on a mesh:

For my use case I’d like to get the color of the sprite at the intersection with a picking ray.

peterimg · October 6, 2022, 11:03am

Hi @Censor

It’s funny cause I’m replaying about this issue, that I opened, and its solution a second time today

I do not know if will it work with sprites. What it gives you when you try?

@Evgeni_Popov might give you some more insight on that He helped me a lot along the years Big shout tout to him

Regards
Peter

Evgeni_Popov · October 6, 2022, 1:07pm

You could copy what the sprite manager is doing to check picking is really inside a non alpha-0 pixel (when sprite.useAlphaForPicking=true):

github.com

BabylonJS/Babylon.js/blob/master/packages/dev/core/src/Sprites/spriteManager.ts#L377-L410


      
          private _checkTextureAlpha(sprite: Sprite, ray: Ray, distance: number, min: Vector3, max: Vector3) {

              if (!sprite.useAlphaForPicking || !this.texture) {

                  return true;

              }

          

              const textureSize = this.texture.getSize();

              if (!this._textureContent) {

                  this._textureContent = new Uint8Array(textureSize.width * textureSize.height * 4);

                  this.texture.readPixels(0, 0, this._textureContent);

              }

          

              const contactPoint = TmpVectors.Vector3[0];

          

              contactPoint.copyFrom(ray.direction);

          

              contactPoint.normalize();

              contactPoint.scaleInPlace(distance);

              contactPoint.addInPlace(ray.origin);

          

              const contactPointU = (contactPoint.x - min.x) / (max.x - min.x) - 0.5;

This file has been truncated. show original

You simply need to replace the end of the function with:

        const r = this._textureContent![(u + v * textureSize.width) * 4 + 0];
        const g = this._textureContent![(u + v * textureSize.width) * 4 + 1];
        const b = this._textureContent![(u + v * textureSize.width) * 4 + 2];
        const alpha = this._textureContent![(u + v * textureSize.width) * 4 + 3];

The calling code is inside the intersects function and let you know how the parameters to the _checkTextureAlpha call are computed:

github.com

BabylonJS/Babylon.js/blob/master/packages/dev/core/src/Sprites/spriteManager.ts#L420-L472


      
          public intersects(ray: Ray, camera: Camera, predicate?: (sprite: Sprite) => boolean, fastCheck?: boolean): Nullable<PickingInfo> {

              const count = Math.min(this.capacity, this.sprites.length);

              const min = Vector3.Zero();

              const max = Vector3.Zero();

              let distance = Number.MAX_VALUE;

              let currentSprite: Nullable<Sprite> = null;

              const pickedPoint = TmpVectors.Vector3[0];

              const cameraSpacePosition = TmpVectors.Vector3[1];

              const cameraView = camera.getViewMatrix();

              let activeRay: Ray = ray;

              let pickedRay: Ray = ray;

          

              for (let index = 0; index < count; index++) {

                  const sprite = this.sprites[index];

                  if (!sprite) {

                      continue;

                  }

          

                  if (predicate) {

                      if (!predicate(sprite)) {

This file has been truncated. show original

Censor · October 6, 2022, 6:57pm

Ok thanks both for your replies, I’ll try and see if that works for me. Will get back to you!

Censor · October 28, 2022, 3:20pm

I tried getting this to work, but so far no luck.
I tried overriding the intersects method for setting a property on PickingInfo which included a Color4 with the color channels. I keep getting back each value as 0…

In my case, I have a pickingInfo on a sprite before I need to check the color. Is it possible to easily do this with the pickingInfo data instead?

Topic		Replies	Views
Get Texture color value at ray collision point Questions	3	815	February 26, 2021
Can I retrieve information from ray casting? Questions	4	518	October 6, 2022
Is there a getTextureCoordinates method that does not require pickWithRay? Questions	5	593	January 17, 2020
Get the coordinates from the intersection between a line and a mesh Questions	7	1210	December 29, 2021
How do I sample a texture at (u, v) on the CPU? Questions	3	489	June 7, 2023

Get color of sprite from ray pickingInfo

Related topics