trident-job / delphi-code-coverage Goto Github PK

When you use generics in your code coverage generated for some units that 
should not be covered and not generated for some units that should be covered

What steps will reproduce the problem?

First scenario
1. Use standard generic collection in unit that you are testing
2. In .map file it will look like this

Line numbers for uYourUnit(System.Generics.Collections.pas) segment .text

In this case breakpoints are set and coverage is generated for 
System.Generics.Collections.pas but they should not because this unit is not 
specified in parameters for coverage.

Second scenario
1. Create your own generic class
2. Create unit tests in which you instantiate this class
3. In .map file it will look like this

Line numbers for uYourUnitTest(YourGenericClass.pas) segment .text

In this case coverage for YourGenericClass.pas is not generated because 
uYourUnitTest is not included in parameters, but it actually should be 
generated because YourGenericClass.pas is included in parameters for coverage.

svn checkout 
http://delphi-code-coverage.googlecode.com/svn/trunk/delphi-code-coverage-read-o
nly

brings:

svn: URL 
'http://delphi-code-coverage.googlecode.com/svn/trunk/delphi-code-coverage-read-
only' doesn't exist

Both at command line and with TortoiseSvn

In the DPR, there a TDebugger object is created but never freed.

Fix:

begin
  try
    Debugger := TDebugger.Create;
    try
      Debugger.start();
    finally
      Debugger.Free;
    end;
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.message);
  end;
end.

The statistics HTML for a single unit is not formatted correctly.


Please provide any additional information below.

In TCoverageReport.AddStatistics method, the "</td>" text is missing the 
closing '>' character - all six of them.

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?


Please use labels and text to provide additional information.

They differ for all units. Percent of covered lines in XML is much higher than 
in HTML report. Example:

<srcfile name="ShellControls.pas"> 
  <coverage value="100% (12/12)" type="class, %"/> 
  <coverage value="100% (371/371)" type="method, %"/> 
  <coverage value="60% (4341/7176)" type="block, %"/> 
  <coverage value="60% (4341/7176)" type="line, %"/>


Unit Name   Number of covered lines     Number of lines (which generated code) 
    Percent(s) covered
ShellControls   749     3549    21%

CodeCoverage fails when using simple '\' to feeds -sd option.
I must give \\ in the folder path.

Either add compatibility with single '\', or add a comment in the documentation.

What steps will reproduce the problem?
1. Generage coverage report
2. Load into CruiseControl.NET
3. View Emma report

What is the expected output? What do you see instead?

I would expect a coverage report to be generated. Nothing is generated/

What version of the product are you using? On what operating system?

Version 0.4

Please provide any additional information below.

The coverage XML report needs to have the summary of the entire project in the 
<all name="all classes"> part, otherwise the emma.xsl report doesn't detect it.

Example output is as follows:

<report>
<stats>
<packages value="8"/>
<classes value="185"/>
<methods value="2061"/>
<srcfiles value="62"/>
<srclines value="13135"/>
</stats>
<data>
<all name="all classes">
<!-- not having this means the report isn't generated -->
<coverage type="class, %" value="100% (185/185)"/>
<coverage type="method, %" value="65%  (1345/2061)"/>
<coverage type="block, %" value="60%  (44997/74846)"/>
<coverage type="line, %" value="64%  (8346.3/13135)"/>
<!-- not having this means the report isn't generated -->
<package name="com.sun.tools.javac.v8.resources">

What steps will reproduce the problem?
1. run codecoverage.exe without parameters

What is the expected output? What do you see instead?
-m switch is listed as mandatory, should be optional

attached patch fixes it

What steps will reproduce the problem?
This is my command line:
CodeCoverage -m Tests.map -e Tests.exe -u ShellBrowser ShellControls 
JamControls JamDialogs Shell_Win32 ShellLink ThreadPool JamWorkItems 
JAmSelectionList -sd ..\Source -od CodeCoverageReport -xml -html

What is the expected output? What do you see instead?
Access violation at address 004D68A0 in module 'CodeCoverage.exe'. Read of 
address 00000000

What version of the product are you using? On what operating system?
V1.0 RC1 on Windows 7 x64 with Delphi XE2

Please provide any additional information below.
I can supply Tests.Exe and Tests.map if you like

Branch name: magicmonty

Purpose of code changes on this branch:

* Refactoring and Cleanup 
* Removed a lot of memory leaks
* Added Compatibility with Delphi 2009

When reviewing my code changes, please focus on:

Please check, if the Code runs on XE and higher

After the review, I'll merge this branch into:
/trunk

Please turn off mem leak reporting in release builds.

See attachment for reported memory leaks.

Version: 1.0 RC8

What steps will reproduce the problem?
1. Run
2. Configure
3. Run Coverage

What is the expected output? What do you see instead?

Standard oput. I still got to toggle to Delphi IDE, do changes, compile and get 
back to this tool...

What version of the product are you using? On what operating system?

RC1

Please provide any additional information below.

Wrell, imho change is straight forward and requiers only CreateProcess() for 
particular dcc32.xe .. :)

The new code in version 0.2 correctly reads the parameter arguments from the 
command line. However, in tests with Delphi 2009, the executable does not 
receive the passed parameters. ParamCount always returns 0.

I have added a debug line in StartProcessToDebug and changed the types to 
PWideChar. Maybe there is a problem with CreateProcess? I am using jwa 2.3 from 
June 25.



function TDebugger.StartProcessToDebug(executable: string): Boolean;
var
  StartInfo: TStartupInfo;
  ProcInfo: TProcessInformation;
  CreateOK: Boolean;
  exe, parameters: PWideChar;
begin
  exe := pwidechar(executable);
  parameters := Pwidechar(Configuration.getApplicationParameters);
  writeln('passing: ' + parameters);

  fillchar(StartInfo, sizeof(TStartupInfo), #0);
  fillchar(ProcInfo, sizeof(TProcessInformation), #0);
  StartInfo.cb := sizeof(TStartupInfo);
  CreateOK := CreateProcess(exe, parameters, nil, nil, false,
    CREATE_NEW_PROCESS_GROUP + NORMAL_PRIORITY_CLASS + DEBUG_PROCESS, nil, nil, StartInfo, ProcInfo);
  result := CreateOK = true;
end;

What steps will reproduce the problem?
1. Run
2. Configure
3. Run Coverage

What is the expected output? What do you see instead?

In-depth BPL analysis

What version of the product are you using? On what operating system?

RC1

Please provide any additional information below.

What steps will reproduce the problem?
1. Run NSIS-based installer with nsisdbg plugin enabled
2. Use Installer
3. Debug bugs or erros

What is the expected output? What do you see instead?

Expected output was possibility to trace and see executed plugin code.

Please use labels and text to provide additional information.

In order to capture output from the process (such as understanding whether a 
unit test is passing) we need to capture stdout and stderr

Each HTML  generated page have CSS information embedded.
This could be extracted to a global CSS file.

What steps will reproduce the problem?
1. Create a test project for a new function that makes use of TDictionary (part 
of System.Generics.Collections delphi library).
2. write a test for that function.
2. run the test coverage tool (RC7 or RC8) with the following parameters: 
-ife -m project1.map -e project1.exe -u coh.utils.myunit -sd 
c:/coh/source/utils -html 

What is the expected output? What do you see instead?
You should see a 100% code coverage result for the only function in the myunit 
unit.
Instead the code coverage tool will report an "Exception during generation of 
unit coverage for:System.Generics.Defaults.pas exception:I/O error 103" error, 
and the code coverage will be at 99% or a similar value. Also, the report will 
include the Generics.defaults.pas library with a 0% code coverage.

What version of the product are you using? On what operating system?
This problem is absent if you use RC6, it is present in RC7, RC8.  The 
operating system is Windows 7 pro 64bit. Delphi XE2.

Please provide any additional information below.
I believe that the error can be at Debugger.pas, in r111 where the line:
    if (AModuleList.IndexOf(UnitModuleName) > -1) then
was changed to 
    if (AModuleList.IndexOf(ModuleName) > -1) then
I haven't built the delphi-code-coverage project from source, so I cannot 
assert that this is truly the source of the problem.

Perhaps I don't understand how I am supposed to use this tool, but I am not 
getting useful statistics (the line by line coverage reporting is GREAT though)

What steps will reproduce the problem?
1. Run the code coverage tool against a unit that has hardly any unit tests. 

What is the expected output? What do you see instead?
I expect to see a very low coverage percentage.

Instead, I get a high percentage.

I'll give you a real example.

We have a Unit with 1430 lines. We have very few unit tests that exercise the 
code in this Unit.

The statistics I get back are

Number of lines covered     173
Number of lines with code gen   270
Line coverage in percent    64%

We are only covering a little more than 10% of the lines in this file. If you 
disregard headers, blank space etc I would still say we only have 20% coverage. 

The tool is only counting lines that "have code gen". I don't see the point of 
this: if a function is not covered by unit tests, it will be dead code, and 
won't appear in the .map file. Isn't this what we should be looking for, and 
adding to our unit tests?? Not just generated code paths that are not called?

This has been my experience of other code coverage tools in other languages.

What version of the product are you using? On what operating system?
0.5

What is the expected output? What do you see instead?

When a DPR is listed in the command line for code coverage, an error is produce:

Exception during generation of unit coverage for:<unit> could not open: 
<unit>.pas


What version of the product are you using?

0.3


Please provide any additional information below.

In the method TCoverageReport.generateUnitReport the following line, when 
added, corrects the issue:

    if not FileExists(sourceFilename) then
      sourceFilename := ChangeFileExt(sourceFilename, '.dpr');

This is placed before "AssignFile(InputFile, sourceFilename);" line.

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?


Please use labels and text to provide additional information.

1/ if the "-m" switch is not present use <name>.map where <Name> is the name of 
the exe in "-m" switch. So : 

CodeCoverage -e TestApp.exe -u TestUnit
is equivalent as :

CodeCoverage -e TestApp.exe -m TestApp.map -u TestUnit

2/ use wildcard in "-u" switch ie :
-u *.pas = all the '.pas' files in the directory.

Is it possible to extend to all major versions ie : D7 -> Xe2 ?

It is not possible to check up the module if his name contains a dot separator 
(Common.Encodings.pas)

What steps will reproduce the problem?
1. run a coverage test where the unit isn't in the current directory
2. if you specify the unit name only on the command line, an error is thrown 
(unit source not found)
3. if you specify the unit's relative path name, it is ignored, and the unit 
isn't marked for coverage.

What is the expected output? What do you see instead?

Expecting coverage information for the unit, but I see either nothing, or only 
the summary.

What version of the product are you using? On what operating system?

0.4

Please provide any additional information below.

Should provide ways to specify unit's paths other than the current paths, as it 
stands, it can only give coverage for the unit test units, but not for the 
units tested by the unit test (which are in various other directories, which 
are too numerous to put in the PATH environment variable).

What steps will reproduce the problem?
Compile following project in Delphi XE2 (not sure about older versions)
program Project1;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils,
  Classes,
  Generics.Defaults,
  Generics.Collections;

var
  Dictionary: TObjectDictionary<integer, TComponent>;
begin
  try
    Dictionary := TObjectDictionary<integer, TComponent>.Create;
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
end.

Run code coverage as 
CodeCoverage.exe -e Project1.exe -u Project1 -sd "C:\Documents and 
Settings\Administrator\My Documents\RAD Studio\Projects\test2\"  -lt

What is the expected output? What do you see instead?

got exception
Exception during generation of unit coverage for:System.Generics.Defaults.pas 
could not open:System.Generics.Defaults.pas
Current directory:C:\Documents and Settings\Administrator\My Documents\RAD 
Studio\Projects\test2
Exception during generation of unit coverage for:System.Generics.Defaults.pas 
exception:I/O error 103

With attached patch it will ignore units that are not specified in -u switch

1. create a simple project with 100% cover and a \test\ directory for the tests
2. create a cover.bat in \test\ to run CodeCoverage.exe 
3. run the cover.bat 

My project is 100% cover and when I run the codeCoverage in the project 
directory all is OK, when I run in \test\ directory my result is 0% code 
coverage
I Tried the following cover.bat :
CodeCoverage -m "..\project1.map" -e "..\project1.exe" -u "..\Unit1"

CodeCoverage -m "C:\CodeCoverage\project1.map" -e 
"C:\CodeCoverage\project1.exe" -u "C:\CodeCoverage\Unit1"

Environnement : D7, XP, CC 0.5

What steps will reproduce the problem?
1.Run CodeCoverage.exe without parameters


What is the expected output? What do you see instead?
Error message is No map file was specified when actually it should be No 
executable was specified

Attached patch fixes it

What steps will reproduce the problem?
1. Launch tool
2. Configure tool ( with wizard ... )
3. Run Coverage...

What is the expected output? What do you see instead?

Ability to have seperate tehncial output about "dead memory" areas. The "dead 
memory" would stand for all the occupied memory bu variables, constants and 
other data structures, that are not:

 - freed ( label "F" )
 - used anywhere in the code ( label "U" )
 - initialized ( labeled "I" )

What version of the product are you using? On what operating system?

RC1

Please provide any additional information below.

To add this feature, the following TODO could be used:

1. On each statement ( <if..then..[begin..end]> ; <for..to..do..[begin..end]> ; 
<repeat..unti> ; and others... ) record used variables into data structure;
2. After code is scanned / covered, perform static code scan and compare if the 
variables used ihn these statements mentioned above exposes the same variables 
declared in the unit / project. If not, provide new column in output labeled 
"Dead Memory", which would describe the main issue that memory is not anymore 
fully accessible for the program...
3. scan the code and combine the found issues in the form of array, that would 
make a columns fiel look like "F,U,I", saying, that the scanned code / 
statement block has got unfreed memory, uninitialized variables and also 
variables are unused ( declared but no value assigned ).

Module name:uRevisionInjector did not match module name:JclResources from 
Addr:0011F227

What does it mean ? Is is an error or a warning ?

it would be helpful to support a new switch "-a" which can be used to pass 
arguments to the executable

Example:

CodeCoverage -m TestApp.map -e TestApp.exe -a /arg1 /arg2=paramvalue -u 
TestUnit TestUnit2

If the Delphi source contains an '<' character, this character won't be 
filtered to &lt; in the HTML output, which will corrupt the output.

any solution for this problem,
please can me for solution

how to display a binary image matrix in a stringrid on delphi

Covered / non covered lines colors, on the htlm detail, are 'light green' and 
'light blue'. They are, in some PC, too close to quickly identifie the non 
covered lines.
I would suggest the red based color for non covered lines.

Attached patch adds two switches
 -dproj delphi-project-file
reads Exename and list of units from Delphi project file

-esm excludemask1 excludemask2 ...
allows to specify masks for source files that should be excluded from coverage

This will allow to run coverage on CodeCoverageTests.exe for example as 
following:
build\CodeCoverage.exe -dproj CodeCoverage\Test\CodeCoverageTests.dproj -esm 
*CodeCoverage\Test\* -od reports\coverage -lt

What steps will reproduce the problem?
1.Enable detailed map file and disable debug information in project options
2.Run codecoverage on it


What is the expected output? What do you see instead?
Should show error message that debug information is missing in map file, 
instead it 
crashes with arithmetic error.

Attached patch fixes it

The TUnitCoverage objects in the private unitlist variable is not freed, nor is 
unitlist.

Fix:

destructor TCoverage.Destroy;
var
  lp : Integer;
begin
  for lp := 0 to Pred(unitlist.Count) do
    unitlist[lp].Free;

  unitlist.Free;
  inherited;
end;

What steps will reproduce the problem?

1. Create a package containing source code (Source.bpl)
2. Create a test executable testing the source code, accessing the source code 
as runtime package (Test.exe)
3. Run CodeCoverage.exe: CodeCoverage.exe -e Test.exe -m Source.map -html

Some parameters missing in 3, but you should get the point.

What is the expected output? What do you see instead?
I would expect to see a code coverage report for the units within Source.bpl. 
Instead, I occur different errors (depending on the package I wanted to create 
a report for):

1) An error message when loading kernel32.dll
2) CodeCoverage.exe doesn't stop loading, logfile is growing and growing (I 
killed execution around ~7 GB).

I attached the logfiles of both errors.

What version of the product are you using? On what operating system?
1.0 RC8, Windows 7 Professional 64bit, Delphi XE2

What steps will reproduce the problem?

1. Build an existing executable with map file
2. Run the executable without the parameter for logfile creation (-lt)

What is the expected output? What do you see instead?

A "Delphi-Code-Coverage-Debug.log" has been created in the output directory (as 
described by the help). In my opinion it would be reasonable to skip logfile if 
parameter -lt is missing. That is:

1) -lt is missing => no logfile
2) -lt was set without filename => default logfile
3) -lt was set with filename => use filename for log

What version of the product are you using? On what operating system?

1.0 RC8, Windows 7 Professional 64bit

if the unit contains dots in it's name, like "Generics.Collections" the unit 
seems to be ignored. It's the same if the unit is specified with -u or -u flags

What steps will reproduce the problem?
1. Create a blank project with a form and a button, on click, ad a showmessage
2. save the unit with "test.unit1.pas" name
3. run codecoverage.exe

What is the expected output? What do you see instead?
the report should specified some data for test.unit1.pas but it's blank

What version of the product are you using? On what operating system?
CodeCoverage_1.0_RC1.

Please provide any additional information below.
  I did not investigated further but during my tests the only time I specified -xml output it complains about a floating point error

Feature Request: would like to see a possibility to load the excluded files 
(parameter -esm) from a file, as it is possible for units (-uf) and source 
directories (-spf)

By the way: -esm is not listed on project home.

What steps will reproduce the problem?

1. Created a project (myproject.dpr) in c:\my_project folder
2. Created a project test (my_project_test.dpr) in c:\my_project folder
3. The sources units are in c:\my_project\source folder
4. The sources (for the tests) are in c:\my_project\test folder
5. The source code for the my_project_test.dpr is similar to:

program MyProjectTest;
uses
   unit1 in 'source\unit1.pas',
   unit2 in 'source\unit2.pas',
   etc.... 

6. Execute the CodeCoverage and the report will be generated in Hudson/Jenkins 
(Continuos Integration). The command used to generate the report is similar to:

CodeCoverage -m my_project_test.map -e my_project_test.exe -od report\coverage 
-uf units-to-coverage.txt

The contents for the "units-to-coverage.txt" are similar to:

unit1
unit2
unit3
etc...


What is the expected output? What do you see instead?

The link (in coverage report - Jenkins) for the unit1 is displayed as 
"source\unit1" and when I click in it occurs an error: 404. This error does not 
occur when the files (uni1.pas, unit2.pas, etc) are in the same directory as 
the my_projet_test.dpr


What version of the product are you using? On what operating system?

- Delphi 7
- Windows Vista
- CodeCoverage 0.5

Thank you

Clayton

The TDebugThread objects in the private mThreadList (:TList) are not freed, nor 
is mThreadList.

Fix:

destructor TDebugProcess.Destroy;
var
  lp : Integer;
begin
  for lp := 0 to Pred(mThreadList.Count) do
    TDebugThread(mThreadList[lp]).Free;

  mThreadList.Free;
  inherited;
end;

Hi

Can you add a switch to generate unmerged emma meta data (.em) and coverage 
data (.ec) files? It's necessary to have report in seperate files if want to 
use Emma plugin to Sonar:
http://docs.codehaus.org/display/SONAR/Sonar+Emma+Plugin
session data (.es) files are not supported

I couldn't find feature allowing for unmerge .es to .ec and .em in Emma.
I'm not sure, but I think, that all you need to do is change several lines in 
EmmaCoverageFileUnit.pas, where emma file is created.
I tried to do it this way and I think it works well. But i will be glad if i 
will be able to use newer versions of Delphi Code Coverage without modification 
every time.

I'm impressed with your work. I think if it will be possible to generate 
Sonar-compatible reports, it will bring you many new users.
Bartholomew

What steps will reproduce the problem?
1. Install tool
2. Configure tool
3. Run coverage

What is the expected output? What do you see instead?

The callstack log created by madexcept would be parsed and highlighted into 
Coverage.

Please use labels and text to provide additional information.

There is several approaches:
  * Use created bugreport.txt
  * Use madExcept source and get more detailed log, but make it more depended on the stability of madExcept. Upgrade madExcept may cause some troubles for tool, if no version check will be implemented.

The command line parameter -od  does not generate an error at startup but also 
does not create the output directory at the end

The following private variables are not freed:
  ms: TJCLMapScanner;
  Configuration: TCoverageConfiguration;
  bplist: TBreakpointList;
  process: TDebugProcess;

Also the TCommandLineProvider object created in the TDebugger.Create is not 
freed.

Fix:

destructor TDebugger.Destroy;
begin
  ms.Free;
  Configuration.Free;
  Coverage.Free;
  bplist.Free;
  process.Free;
  inherited;
end;

There can be lines in the code who will never be covered, because of spurious 
compiler code generation or other special code situations.

One possible solution would be to detect special comments on a line, and when 
encountered, not report the line as not covered.

For instance in:

   procedure MyProc;
   begin
      ThisProcedureAlwaysTriggersAnException;
   end; {expected coverage issue}

the compiler will generate code for the 'end;', which can never be reached.
Other than spurious codegens, such cases can happen in other optimization 
scenarios where the compiler doesn't keep up.