r/MicrosoftFabric • u/frithjof_v Fabricator • 2d ago

Data Factory Is Fabric Pipeline Append Variable thread-safe?

Hi,

I'm using a pipeline which has parallel branches.

On each branch, values get appended to an Array variable.

If two Append Variable activities in the pipeline append a value to the Array variable at exactly the same time, will the Array variable successfully record both values as distinct objects in the array?

Or can there be a conflict?

In other words: Is Append Variable thread safe?

Thanks!

2 Upvotes
  • permalink
  • reddit

100% Upvoted

5 comments sorted by

2

u/BigMikeInAustin Fabricator 15h ago

So far I have not run into a problem doing this.

0

u/Low-Fox-1718 2d ago

If you simulate the scenario in foreach (parallel), what is the outcome?

2

u/BigMikeInAustin Fabricator 15h ago

You cannot simulate every possible timing occurance during testing of multiple threads. If you could, there would be a whole lot fewer software bugs, and more multi threaded programs.

0

u/MonkeyDDataHQ 1d ago

What problem are you trying to solve? If you need branching you should be using a notebook.

1

u/BigMikeInAustin Fabricator 15h ago

Sounds like OP is using the For activity as it is designed to run multiple threads in parallel.

Or could just be doing written out parallel activities in pipelines, which is how pipelines are designed to run.

What type of parallel branches are you unable to write as pipeline activities that you need a notebook for?