Fix: Omit tests directory from code coverage

[sauravsharma1998]

No description provided.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.4%. Comparing base (294dcd0) to head (80b5d84).
Report is 14 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##           master    #735      +/-   ##
=========================================
+ Coverage    61.5%   89.4%   +27.9%     
=========================================
  Files          43      22      -21     
  Lines        2786    1122    -1664     
  Branches      161       0     -161     
=========================================
- Hits         1713    1002     -711     
+ Misses       1060     120     -940     
+ Partials       13       0      -13     

Flag	Coverage Δ
mypy	?
tests	89.4% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[WisdomPill]

why do we need these ignores?

[WisdomPill]

please restore this, as also tests should be checked for typing errors so that we can notice breaking changes

[sauravsharma1998]

Sure @WisdomPill ! This was just experimental. Any suggestions to avoid tests directory from coverage report?

[WisdomPill]

also changelog is missing

[sauravsharma1998]

@WisdomPill Added new config file codecov.yml to manage the coverage on the basis of flags and directory.

Motivation - https://docs.codecov.com/docs/github-5b-setting-coverage-standards-with-flags

Other way:
We can skip uploading cobertura.xml generated via mypy command for coverage.io
Since coverage.xml will be sufficient for code coverage matrix

[WisdomPill]

we still need coverage on tests for mypy

[WisdomPill]

why not change the codecov in .github folder?

[sauravsharma1998]

@WisdomPill Do we require to upload mypy report on coverage.io?

[sauravsharma1998]

By default coverage.io will merge all the coverages, that's why coverage percentage is lower than expected.

[WisdomPill]

we are using codecov, coverage.io does not exist as far as I know. I was asking why did you not change this file instead and created a new one.

And yes we need to upload the coverage of mypy too to codecov

[sauravsharma1998]

Got your point @WisdomPill !

With coverage.io I meant to https://app.codecov.io/

Could you provide any idea, Why mypy coverage is low? It would be helpful to think further in combining both type of reports.

[sauravsharma1998]

MyPy's type checking does not fall under the category of code coverage.

Code coverage typically refers to the measurement of how much of your codebase is exercised by your test suite during execution. It's a metric that indicates the percentage of lines, branches, or other code elements that have been executed by your tests. Code coverage tools like coverage.py or JaCoCo track which parts of your code are executed during testing and generate reports to help you understand how thoroughly your tests exercise your code.

On the other hand, MyPy is a static type checker for Python code. It analyzes your codebase to identify potential type-related issues, inconsistencies, or violations of type hints. MyPy operates statically, meaning it analyzes the code without actually running it. Instead of measuring code coverage, MyPy focuses on ensuring type safety and correctness by enforcing type annotations and performing type inference.

@WisdomPill Please correct me if I am wrong.

[sauravsharma1998]

@WisdomPill Instead of uploading mypy report to codecov.io we can generated html formatted report of mypy(Which will satisfy purposes for both code coverage and type checking coverage)

Sample Screenshot:

mypy report can be published via smokeshow(https://smokeshow.helpmanual.io/)
Sample hosted report link - https://smokeshow.helpmanual.io/1w0y4w4i371c586h5a3f/

If it's okay for you, we can proceed. Else have to find another way.