Fast TensorAccessor by haytham2597 · Pull Request #1396 · dotnet/TorchSharp

haytham2597 · 2024-10-25T16:33:31Z

No description provided.

TorchSharp.sln

src/TorchSharp/Utils/TensorAccessor.cs

…TorchSharp into fast_tensor_accesor

test/TorchSharpTest/TestJIT.cs

test/TorchSharpTest/TestTorchTensor.cs

haytham2597

Fixed some error on my FastTensorAccessor and rollback SLN file to Original of this Repo

alinpahontu2912 · 2025-03-26T09:25:20Z

Hey @haytham2597, are you still intending to work on this PR?

haytham2597 · 2025-03-26T14:53:59Z

Hey @haytham2597, are you still intending to work on this PR?

This PR is already Finish. At least i know is worked and is fast.

alinpahontu2912 · 2025-04-16T07:29:11Z

Hey @haytham2597, can you add a releasenotes comment, rebase and solve the conflict to get this merged ?

…_tensor_accesor

haytham2597

@alinpahontu2912 Finished implement of Fast TensorAccesor with current update main i think i need wait for approving of @NiklasGustafsson for merge.

…TorchSharp into fast_tensor_accesor

alinpahontu2912 · 2025-07-02T09:40:10Z

Hey @haytham2597, Niklas is no longer working on the project. I have taken over. This seems like it was reviewed by him before, so if tests pass, I can merge it.

haytham2597 · 2025-07-16T19:26:30Z

Hey @haytham2597, Niklas is no longer working on the project. I have taken over. This seems like it was reviewed by him before, so if tests pass, I can merge it.

Nice, i think this test passed very well. I don't have any new upgrade/change for this so you can freely merge.
But i believe that need run test for the current branch.

Copilot

Pull request overview

This PR introduces a “fast” path for TensorAccessor<T> array extraction/copying, adds tests intended to validate the new behavior, and updates release notes / test tolerances accordingly.

Changes:

Reworked TensorAccessor<T>.ToArray(), ToNDArray(), CopyTo(), and CopyFrom() with new pointer-based implementations and a new ToArray(from_index, count) overload.
Added new unit tests covering TensorAccessor<T>.ToArray() and copy methods on contiguous/non-contiguous tensors.
Relaxed a JIT numerical comparison tolerance and documented the TensorAccessor change in release notes.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
`src/TorchSharp/Utils/TensorAccessor.cs`	Adds new “fast” implementations for array extraction and copy operations.
`test/TorchSharpTest/TestTorchTensor.cs`	Adds tests targeting the new fast TensorAccessor behaviors.
`test/TorchSharpTest/TestJIT.cs`	Adjusts an `allclose` assertion tolerance.
`RELEASENOTES.md`	Notes the “Fast TensorAccessor” change.
`.gitignore`	Adds ignore rules for `TestClear`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-13T13:37:37Z

src/TorchSharp/Utils/TensorAccessor.cs

+            if (_tensor.is_contiguous()) {
+                ToArray().CopyTo(array);
+                return;
            }
+            ToArray().CopyTo(array);


CopyTo(Span<T> array, int arrayIndex = 0, long tensorIndex = 0) ignores both arrayIndex and tensorIndex (it calls ToArray().CopyTo(array)), and the contiguous fast path still allocates a full intermediate array. This should copy directly from the tensor starting at tensorIndex into array.Slice(arrayIndex, ...) without allocating.

Suggested change

if (_tensor.is_contiguous()) {

ToArray().CopyTo(array);

return;

}

ToArray().CopyTo(array);

if (arrayIndex < 0 || arrayIndex > array.Length)

throw new ArgumentOutOfRangeException(nameof(arrayIndex));

long count = Count;

if (tensorIndex < 0 || tensorIndex > count)

throw new ArgumentOutOfRangeException(nameof(tensorIndex));

if (_tensor.is_contiguous()) {

long availableFromTensor = count - tensorIndex;

int availableInArray = array.Length - arrayIndex;

int maxCount = Math.Min((int)availableFromTensor, availableInArray);

if (maxCount <= 0)

return;

unsafe {

// Source pointer: start at tensorIndex

T* srcPtr = (T*)_tensor_data_ptr;

srcPtr += tensorIndex;

// Destination: slice the span to the desired region

var destSlice = array.Slice(arrayIndex, maxCount);

fixed (T* destPtr = &MemoryMarshal.GetReference(destSlice)) {

long byteCount = (long)maxCount * sizeof(T);

Buffer.MemoryCopy(srcPtr, destPtr, byteCount, byteCount);

}

}

return;

}

// Non-contiguous tensor: copy element-by-element without allocating an intermediate array.

if (tensorIndex >= count || arrayIndex >= array.Length)

return;

int written = 0;

long skipped = 0;

foreach (var value in this) {

if (skipped < tensorIndex) {

skipped++;

continue;

}

int targetIndex = arrayIndex + written;

if (targetIndex >= array.Length)

break;

array[targetIndex] = value;

written++;

if (written >= array.Length - arrayIndex)

break;

}

Copilot · 2026-02-13T13:37:38Z

src/TorchSharp/Utils/TensorAccessor.cs

        public void CopyFrom(T[] array, int arrayIndex = 0, long tensorIndex = 0)
        {
-            int idx = arrayIndex;
-            foreach (int offset in GetSubsequentIndices(tensorIndex)) {
-                if (idx >= array.Length) break;
-                unsafe { ((T*)_tensor_data_ptr)[offset] = array[idx]; }
-                idx += 1;
-            }
+            SetValueTensor(ref array, _tensor.shape, _tensor.stride(), Count, arrayIndex, onThis:true);
        }


CopyFrom(T[] array, int arrayIndex = 0, long tensorIndex = 0) passes arrayIndex into SetValueTensor() as if it were the starting tensor index, and ignores tensorIndex. This changes semantics vs. the old implementation (array offset != tensor offset) and can write into the wrong tensor positions.

Copilot · 2026-02-13T13:37:38Z

src/TorchSharp/Utils/TensorAccessor.cs

+                for (long index = arrayIndex; index < count; index++) {
+                    long offset = index;
+                    long ptrIndex = 0;
+                    for (long d = shape.Length - 1; d >= 0; d--) // Traverse dimensions in reverse order
+                    {
+                        long i = offset % shape[d]; // Current index in dimension d
+                        ptrIndex += i * strides[d]; // Calculate ptrIndex using strides
+                        offset /= shape[d]; // Move to the next dimension
+                    }
+                    ptr[ptrIndex] = array[(int)index];


CopyFrom(ReadOnlySpan<T> array, int arrayIndex = 0, long tensorIndex = 0) uses the loop variable index as both the tensor linear index and the source span index (array[(int)index]), which will go out of range whenever Count > array.Length and also ignores tensorIndex. It should map source index src = index - tensorIndex + arrayIndex and limit the loop to the min of source/destination remaining lengths.

Suggested change

for (long index = arrayIndex; index < count; index++) {

long offset = index;

long ptrIndex = 0;

for (long d = shape.Length - 1; d >= 0; d--) // Traverse dimensions in reverse order

{

long i = offset % shape[d]; // Current index in dimension d

ptrIndex += i * strides[d]; // Calculate ptrIndex using strides

offset /= shape[d]; // Move to the next dimension

}

ptr[ptrIndex] = array[(int)index];

if (arrayIndex < 0 || arrayIndex > array.Length)

throw new ArgumentOutOfRangeException(nameof(arrayIndex));

if (tensorIndex < 0 || tensorIndex > count)

throw new ArgumentOutOfRangeException(nameof(tensorIndex));

long maxSrc = array.Length - arrayIndex;

long maxDst = count - tensorIndex;

long elementsToCopy = Math.Min(maxSrc, maxDst);

for (long i = 0; i < elementsToCopy; i++) {

long linearIndex = tensorIndex + i;

long offset = linearIndex;

long ptrIndex = 0;

for (long d = shape.Length - 1; d >= 0; d--) // Traverse dimensions in reverse order

{

long dimIndex = offset % shape[d]; // Current index in dimension d

ptrIndex += dimIndex * strides[d]; // Calculate ptrIndex using strides

offset /= shape[d]; // Move to the next dimension

}

int srcIndex = arrayIndex + (int)i;

ptr[ptrIndex] = array[srcIndex];

Copilot · 2026-02-13T13:37:38Z

test/TorchSharpTest/TestTorchTensor.cs

 // Copyright (c) .NET Foundation and Contributors.  All Rights Reserved.  See LICENSE in the project root for license information.
 using System;
 using System.Collections.Generic;
+using System.Diagnostics;


using System.Diagnostics; is added but not used anywhere in this test file; it should be removed to avoid unused-using warnings (and to keep the file consistent with the rest of the using list).

Suggested change

using System.Diagnostics;

Copilot · 2026-02-13T13:37:38Z

test/TorchSharpTest/TestTorchTensor.cs

+                float[] v = t.data<float>().ToArray(4);
+                float[] res = new float[v.Length];
+                res[0] = t[1, 0, 1].item<float>();
+                res[1] = t[1, 0, 2].item<float>();
+                Assert.Equal(res, v);
+            }


The new ToArrayIndexFastTensorAccessor test doesn't assert the expected length of v and leaves trailing entries of res at default values. As written, it can still pass even if ToArray(4) returns extra default elements beyond the two expected values; add an explicit length assertion and/or fully populate res to validate the contract.

Copilot · 2026-02-13T13:37:40Z

src/TorchSharp/Utils/TensorAccessor.cs

        public void CopyTo(T[] array, int arrayIndex = 0, long tensorIndex = 0)
        {
-            int idx = arrayIndex;
-            foreach (int offset in GetSubsequentIndices(tensorIndex)) {
-                if (idx >= array.Length) break;
-                unsafe { array[idx] = ((T*)_tensor_data_ptr)[offset]; }
-                idx += 1;
+            if (_tensor.is_contiguous()) {
+                CopyContiguous(array, arrayIndex, array.Length);
+                return;
            }
+            ToArray().CopyTo(array, arrayIndex);
        }


CopyTo(T[] array, int arrayIndex = 0, long tensorIndex = 0) no longer honors tensorIndex (it always copies from the start of the tensor). This is a behavioral change from the previous implementation and will produce wrong results for callers that request copying from a non-zero tensor offset.

Copilot · 2026-02-13T13:37:40Z

test/TorchSharpTest/TestTorchTensor.cs

+                float[] toCopy = new float[] { 1, 2 };
+                var t = rand(2, 1, 3, ScalarType.Float32);
+                t.data<float>().CopyFrom(toCopy);
+                Assert.True(t[0,0,0].item<float>() == toCopy[0]);


Equality checks on floating point values can yield unexpected results.

Copilot · 2026-02-13T13:37:40Z

test/TorchSharpTest/TestTorchTensor.cs

+                Assert.True(t[0,0,0].item<float>() == toCopy[0]);
+                Assert.True(t[0,0,1].item<float>() == toCopy[1]);
+            }
+            {
+                //With offset
+                float[] toCopy = new float[] { 9, 3 };
+                var t = rand(2, 1, 3, ScalarType.Float32);
+                t.data<float>().CopyFrom(toCopy, 2);
+                Assert.True(t[0, 0, 2].item<float>() == toCopy[0]);
+                Assert.True(t[1, 0, 0].item<float>() == toCopy[1]);


Equality checks on floating point values can yield unexpected results.

Suggested change

Assert.True(t[0,0,0].item<float>() == toCopy[0]);

Assert.True(t[0,0,1].item<float>() == toCopy[1]);

}

{

//With offset

float[] toCopy = new float[] { 9, 3 };

var t = rand(2, 1, 3, ScalarType.Float32);

t.data<float>().CopyFrom(toCopy, 2);

Assert.True(t[0, 0, 2].item<float>() == toCopy[0]);

Assert.True(t[1, 0, 0].item<float>() == toCopy[1]);

Assert.Equal(toCopy[0], t[0, 0, 0].item<float>());

Assert.Equal(toCopy[1], t[0, 0, 1].item<float>());

}

{

//With offset

float[] toCopy = new float[] { 9, 3 };

var t = rand(2, 1, 3, ScalarType.Float32);

t.data<float>().CopyFrom(toCopy, 2);

Assert.Equal(toCopy[0], t[0, 0, 2].item<float>());

Assert.Equal(toCopy[1], t[1, 0, 0].item<float>());

Copilot · 2026-02-13T13:37:40Z

test/TorchSharpTest/TestTorchTensor.cs

+                float[] toCopy = new float[] { 9, 3 };
+                var t = rand(2, 1, 3, ScalarType.Float32);
+                t.data<float>().CopyFrom(toCopy, 2);
+                Assert.True(t[0, 0, 2].item<float>() == toCopy[0]);


Equality checks on floating point values can yield unexpected results.

Copilot · 2026-02-13T13:37:40Z

test/TorchSharpTest/TestTorchTensor.cs

+                Assert.True(t[0,0,0].item<float>() == toCopy[0]);
+                Assert.True(t[0,0,1].item<float>() == toCopy[1]);
+            }
+            {
+                //With offset
+                float[] toCopy = new float[] { 9, 3 };
+                var t = rand(2, 1, 3, ScalarType.Float32);
+                t.data<float>().CopyFrom(toCopy, 2);
+                Assert.True(t[0, 0, 2].item<float>() == toCopy[0]);
+                Assert.True(t[1, 0, 0].item<float>() == toCopy[1]);


Equality checks on floating point values can yield unexpected results.

Suggested change

Assert.True(t[0,0,0].item<float>() == toCopy[0]);

Assert.True(t[0,0,1].item<float>() == toCopy[1]);

}

{

//With offset

float[] toCopy = new float[] { 9, 3 };

var t = rand(2, 1, 3, ScalarType.Float32);

t.data<float>().CopyFrom(toCopy, 2);

Assert.True(t[0, 0, 2].item<float>() == toCopy[0]);

Assert.True(t[1, 0, 0].item<float>() == toCopy[1]);

Assert.Equal(toCopy[0], t[0, 0, 0].item<float>(), 5);

Assert.Equal(toCopy[1], t[0, 0, 1].item<float>(), 5);

}

{

//With offset

float[] toCopy = new float[] { 9, 3 };

var t = rand(2, 1, 3, ScalarType.Float32);

t.data<float>().CopyFrom(toCopy, 2);

Assert.Equal(toCopy[0], t[0, 0, 2].item<float>(), 5);

Assert.Equal(toCopy[1], t[1, 0, 0].item<float>(), 5);

haytham2597 added 2 commits October 25, 2024 13:15

fast copy tensor accessor

ce679e2

rollback sln

958a187

NiklasGustafsson reviewed Oct 25, 2024

View reviewed changes

Niklas Gustafsson and others added 4 commits October 25, 2024 10:15

Merge branch 'main' into fast_tensor_accesor

abe9990

Numel

0b20f13

Merge branch 'fast_tensor_accesor' of https://github.com/haytham2597/…

7df8e46

…TorchSharp into fast_tensor_accesor

original TorchSharp.sln

d6865a6

LukePoga mentioned this pull request Nov 1, 2024

No way to copy a tensor from gpu to cpu to pre allocated array. #1388

Open

haytham2597 added 5 commits November 2, 2024 12:40

Fast Accesor to non contiguous tensor

d2857bf

Test of TensorAccessor Contiguous and Non-Contiguous

f5e43d7

Procedural NDArray

6235075

Fix TestJit_3 Threshold

1ab3891

Improve tensor accessor

19effd7

haytham2597 commented Nov 3, 2024

View reviewed changes

test/TorchSharpTest/TestJIT.cs Show resolved Hide resolved

haytham2597 commented Nov 3, 2024

View reviewed changes

test/TorchSharpTest/TestTorchTensor.cs Show resolved Hide resolved

haytham2597 added 2 commits November 3, 2024 17:51

Simplify

9ff7866

fix index and add more test

9e6ba01

haytham2597 commented Nov 9, 2024

View reviewed changes

haytham2597 requested a review from NiklasGustafsson November 9, 2024 21:42

Niklas Gustafsson added 3 commits November 12, 2024 07:51

Merge branch 'main' into fast_tensor_accesor

46fe73a

Merge branch 'main' into fast_tensor_accesor

d24c709

Merge branch 'main' into fast_tensor_accesor

63aa144

haytham2597 added 3 commits April 16, 2025 12:56

Merge branch 'main' of https://github.com/dotnet/TorchSharp into fast…

c29592b

…_tensor_accesor

release note

c031c61

Update RELEASENOTES.md

0fe2f96

haytham2597 commented Apr 16, 2025

View reviewed changes

alinpahontu2912 self-requested a review April 17, 2025 11:46

haytham2597 and others added 4 commits April 17, 2025 17:14

Merge branch 'main' into fast_tensor_accesor

ac06718

removed unused

1ea4266

Merge branch 'fast_tensor_accesor' of https://github.com/haytham2597/…

5d50ebe

…TorchSharp into fast_tensor_accesor

Merge branch 'main' into fast_tensor_accesor

e55f01c

haytham2597 and others added 2 commits August 2, 2025 22:48

Merge branch 'main' into fast_tensor_accesor

d0e5db8

Merge branch 'main' into fast_tensor_accesor

4304c3f

alinpahontu2912 requested review from Copilot and removed request for NiklasGustafsson February 13, 2026 13:33

Copilot started reviewing on behalf of alinpahontu2912 February 13, 2026 13:33 View session

Copilot AI reviewed Feb 13, 2026

View reviewed changes

-            if (_tensor.is_contiguous()) {
-                ToArray().CopyTo(array);
-                return;
-            }
-            ToArray().CopyTo(array);
+            if (arrayIndex < 0 || arrayIndex > array.Length)
+                throw new ArgumentOutOfRangeException(nameof(arrayIndex));
+            long count = Count;
+            if (tensorIndex < 0 || tensorIndex > count)
+                throw new ArgumentOutOfRangeException(nameof(tensorIndex));
+            if (_tensor.is_contiguous()) {
+                long availableFromTensor = count - tensorIndex;
+                int availableInArray = array.Length - arrayIndex;
+                int maxCount = Math.Min((int)availableFromTensor, availableInArray);
+                if (maxCount <= 0)
+                    return;
+                unsafe {
+                    // Source pointer: start at tensorIndex
+                    T* srcPtr = (T*)_tensor_data_ptr;
+                    srcPtr += tensorIndex;
+                    // Destination: slice the span to the desired region
+                    var destSlice = array.Slice(arrayIndex, maxCount);
+                    fixed (T* destPtr = &MemoryMarshal.GetReference(destSlice)) {
+                        long byteCount = (long)maxCount * sizeof(T);
+                        Buffer.MemoryCopy(srcPtr, destPtr, byteCount, byteCount);
+                    }
+                }
+                return;
+            }
+            // Non-contiguous tensor: copy element-by-element without allocating an intermediate array.
+            if (tensorIndex >= count || arrayIndex >= array.Length)
+                return;
+            int written = 0;
+            long skipped = 0;
+            foreach (var value in this) {
+                if (skipped < tensorIndex) {
+                    skipped++;
+                    continue;
+                }
+                int targetIndex = arrayIndex + written;
+                if (targetIndex >= array.Length)
+                    break;
+                array[targetIndex] = value;
+                written++;
+                if (written >= array.Length - arrayIndex)
+                    break;
+            }

Conversation

haytham2597 commented Oct 25, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

haytham2597 left a comment

Choose a reason for hiding this comment

Uh oh!

alinpahontu2912 commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

haytham2597 commented Mar 26, 2025

Uh oh!

alinpahontu2912 commented Apr 16, 2025

Uh oh!

haytham2597 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alinpahontu2912 commented Jul 2, 2025

Uh oh!

haytham2597 commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alinpahontu2912 commented Mar 26, 2025 •

edited

Loading

haytham2597 left a comment •

edited

Loading

haytham2597 commented Jul 16, 2025 •

edited

Loading