Skip to content

ci: update tst workflow and add supporting scripts#452

Closed
c-fteixeira wants to merge 1 commit intomainfrom
ft/test-runner
Closed

ci: update tst workflow and add supporting scripts#452
c-fteixeira wants to merge 1 commit intomainfrom
ft/test-runner

Conversation

@c-fteixeira
Copy link
Copy Markdown
Collaborator

No description provided.

@c-fteixeira c-fteixeira deleted the ft/test-runner branch July 12, 2024 15:35
vadiklyutiy added a commit that referenced this pull request Dec 19, 2024
…patial(47) (#452)

Introduce new IR optimization pass `spatial_simplification_pass`.
It removes all `1` from `spatial` mapping.

E.g. transform `spatial(1,47)` to `spatial(47)`.

`spatial_simplification_pass` doesn't help a lot right now but would bring perf improvement together with #449
vadiklyutiy added a commit that referenced this pull request Dec 19, 2024
### Performance
 - synthetic test, elementwise add `a = b + 1`. Improvement around 2%

 - models (together with #452 )

|model|latency|prev_latency|delta|
|--------|--------|--------|--------|
|bert-base-uncased|19.6886|20.0891|2.034
|densenet121|35.0828|35.2025|0.341
|efficientnet_b0|18.8482|18.9655|0.622
|mobilenet_v2|11.5672|11.5901|0.198
|resnet50|29.0077|29.6469|2.204
|vit_b_16|123.344|126.705|2.725
|**GMEAN**| | |**1.349**

  
|model|latency|prev_latency|delta|
|--------|--------|--------|--------|
|bert-base-uncased|19.6886|19.9814|1.487
|densenet121|35.0828|35.2032|0.343
|efficientnet_b0|18.8482|18.9711|0.652
|mobilenet_v2|11.5672|11.5913|0.208
|resnet50|29.0077|29.101|0.322
|vit_b_16|123.344|124.837|1.21
|**GMEAN**| | |**0.702**

The above comparisons were done against exactly the same `main` branch.
Yes, we have a big fluctuation in perf results :(


### TODO
- doesn't work with dynamic shapes yet
- only `spatial` mapping is supported right now. Another mapping support
should be investigating
vadiklyutiy added a commit that referenced this pull request Dec 20, 2024
…patial(47) (#452)

Introduce new IR optimization pass `spatial_simplification_pass`.
It removes all `1` from `spatial` mapping.

E.g. transform `spatial(1,47)` to `spatial(47)`.

`spatial_simplification_pass` doesn't help a lot right now but would bring perf improvement together with #449
vadiklyutiy added a commit that referenced this pull request Dec 20, 2024
### Performance
 - synthetic test, elementwise add `a = b + 1`. Improvement around 2%

 - models (together with #452 )

|model|latency|prev_latency|delta|
|--------|--------|--------|--------|
|bert-base-uncased|19.6886|20.0891|2.034
|densenet121|35.0828|35.2025|0.341
|efficientnet_b0|18.8482|18.9655|0.622
|mobilenet_v2|11.5672|11.5901|0.198
|resnet50|29.0077|29.6469|2.204
|vit_b_16|123.344|126.705|2.725
|**GMEAN**| | |**1.349**

  
|model|latency|prev_latency|delta|
|--------|--------|--------|--------|
|bert-base-uncased|19.6886|19.9814|1.487
|densenet121|35.0828|35.2032|0.343
|efficientnet_b0|18.8482|18.9711|0.652
|mobilenet_v2|11.5672|11.5913|0.208
|resnet50|29.0077|29.101|0.322
|vit_b_16|123.344|124.837|1.21
|**GMEAN**| | |**0.702**

The above comparisons were done against exactly the same `main` branch.
Yes, we have a big fluctuation in perf results :(


### TODO
- doesn't work with dynamic shapes yet
- only `spatial` mapping is supported right now. Another mapping support
should be investigating
vadiklyutiy added a commit that referenced this pull request Dec 26, 2024
…patial(47) (#452)

Introduce new IR optimization pass `spatial_simplification_pass`.
It removes all `1` from `spatial` mapping.

E.g. transform `spatial(1,47)` to `spatial(47)`.

`spatial_simplification_pass` doesn't help a lot right now but would bring perf improvement together with #449
vadiklyutiy added a commit that referenced this pull request Dec 26, 2024
### Performance
 - synthetic test, elementwise add `a = b + 1`. Improvement around 2%

 - models (together with #452 )

|model|latency|prev_latency|delta|
|--------|--------|--------|--------|
|bert-base-uncased|19.6886|20.0891|2.034
|densenet121|35.0828|35.2025|0.341
|efficientnet_b0|18.8482|18.9655|0.622
|mobilenet_v2|11.5672|11.5901|0.198
|resnet50|29.0077|29.6469|2.204
|vit_b_16|123.344|126.705|2.725
|**GMEAN**| | |**1.349**

  
|model|latency|prev_latency|delta|
|--------|--------|--------|--------|
|bert-base-uncased|19.6886|19.9814|1.487
|densenet121|35.0828|35.2032|0.343
|efficientnet_b0|18.8482|18.9711|0.652
|mobilenet_v2|11.5672|11.5913|0.208
|resnet50|29.0077|29.101|0.322
|vit_b_16|123.344|124.837|1.21
|**GMEAN**| | |**0.702**

The above comparisons were done against exactly the same `main` branch.
Yes, we have a big fluctuation in perf results :(


### TODO
- doesn't work with dynamic shapes yet
- only `spatial` mapping is supported right now. Another mapping support
should be investigating
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant