Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

complex expressions very slow #19

Open
SiggyRadiation opened this issue Jun 19, 2019 · 1 comment
Open

complex expressions very slow #19

SiggyRadiation opened this issue Jun 19, 2019 · 1 comment

Comments

@SiggyRadiation
Copy link

I have a somewhat complex expression and it causes extreme slowness in cases like these:

  • During editing of the expression, the user interface sometimes locks up for over a minute
  • When exiting the expression editor
  • Even when starting up the Azure Data Factory webapp (nothing opened yet). There is a phase when it is loading the resources in all the data flows that takes very long. This means time from clicking an url to open Azure Data Factory to it being ready to work with can take minutes just to load the resource (see image below).

This is the expression:


iif (instr(code_bekostigingstatus, 'pi') > 0 ,'', 
iif (instr(code_bekostigingstatus, 'jl') > 0, 'Geen 1e inschrijving', 
iif (instr(code_bekostigingstatus, 'nr') > 0, 'Nationaliteiteneis',
iif (instr(code_bekostigingstatus, 'nb') > 0, 'Eerdere graad behaald',
iif (instr(code_bekostigingstatus, 'nd') > 0,'Eerdere graad behaald',
iif (instr(code_bekostigingstatus, 'nh') > 0, 'Bekostigingsjaren verbruikt',
iif (instr(code_bekostigingstatus, 'ni') > 0, 'Bekostigingsjaren verbruikt',
iif (instr(code_bekostigingstatus, 'nk') > 0, 'Bekostigingsjaren verbruikt',
iif (instr(code_bekostigingstatus, 'ti') > 0, 'Geen tijdige aanlevering',
'Anders')))))))))

Fyi the logic behind this is that the field code_bekostigingsstatus can contain multiple codes seperated by a comma. The organisation only wants to know the meaning of most highest priority code that is contained in the field, so I scan the field from highest to lowest priority code.

image

@kromerm
Copy link
Owner

kromerm commented Jul 1, 2019

How many columns are in your source data?

I'd like to have our Engineering team look at this. Can you send us an email directly? Email is [email protected].

Go to the Data Flow designer UI and click the Code button at the top right, then choose "Plan".

Copy & paste that script into the email with your problem description from above.

Thank you!! Best, Mark Kromer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants