Dr_Rick
(Fresh Scripter)
2009-09-22 11:01 PM
Removal of Duplicates in ListView

I am experiencing trouble with this function that I created. I have to execute the function twice to remove all the items that apply. The data inside SubItems(3) is a date string. I am removing older entries in the list by comparing the strings. I have stepped through it many times in debug mode and fail to see what I have to run it twice to catch all the entires.

 Code:
   For $g = 0 To $EndOfList-1
    If($ListViewEx1.Items($g).Subitems(3).text = "")
       $G = $EndofList -1 
    EndIf
     For $j = 0 To $EndOfList -1
      If($ListViewEx1.Items($j).Subitems(3).text = "")
        $j = $EndofList -1
      EndIf 
      $sFirstLine = ($ListViewEx1.Items($g).Subitems(1).text + " " + $ListViewEx1.Items($g).Subitems(3).text)
      $sSecondLine = ($ListViewEx1.Items($j).Subitems(1).text + " " + $ListViewEx1.Items($j).Subitems(3).text)
	  If($ListViewEx1.Items($g).Subitems(3).text) = ($ListViewEx1.Items($j).Subitems(3).text)
         If ($sFirstLine < $sSecondLine) 
          $ListViewEx1.Items.Remove($g)
         EndIf
       EndIf
       If ($ListViewEx1.Items($g).Subitems(3).text) = ($ListViewEx1.Items($j).Subitems(3).text)
        If ($sFirstLine > $sSecondLine)
         $ListViewEx1.Items.Remove($j)
        EndIf
       EndIf
     Next ;J
   Next ;G

Richard H.Administrator
(KiX Supporter)
2009-09-23 09:31 AM
Re: Removal of Duplicates in ListView

Have you considered what happens to the indexes of (later) items when you delete a line?

Let's take a simple list (with indexes in brackets []:
 Quote:
[0] APPLE
[1] BANANA
[2] APPLE
[3] BANANA


On your first pass $g = [0], and $j finds a match for APPLE at [3].

You delete [0] and now your list looks like this:
 Quote:
[0] BANANA
[1] APPLE
[2] BANANA


On your next pass $g = [1]

Oh-oh. You've missed BANANA because it has been shuffled up the list.

If you re-run the code you will of course catch BANANA.

If you can have more than one duplicate of course the problem may be worse.

You can also make a small performance gain - there is no point in starting the inner loop from zero as the first eligible datum is the one following the outer loop index.

Dr_Rick
(Fresh Scripter)
2009-09-24 04:09 AM
Re: Removal of Duplicates in ListView

In rethinking the logic of my loop I find that it fails to remove the duplicates upon execution. I have a log file that is read into the listview. The comparison takes into consideration two columns, one of which is a date string appended with the time. The other column is a unique item number. When the function is called it compares two lines and attempts the removal of the one that was older, leaving the latest entry in the listview. I am at a loss as to why it is not working.

 Code:

  $TabControl1.SelectedIndex = 6 
  $EndOfList = $ListViewEx7.Items.Count
  For $g = 1 To $EndOfList -1
  $sFirstLine = ($ListViewEx7.Items(0).Subitems(1).text + " " + $ListViewEx7.Items(0).Subitems(3).text)
  $sSecondLine = ($ListViewEx7.Items($g).Subitems(1).text + " " + $ListViewEx7.Items($g).Subitems(3).text)
         If ($sFirstLine < $sSecondLine) OR ($sFirstLine = $sSecondLine)
          $ListViewEx7.Items.Remove(0)
         EndIf
  Next


Richard H.Administrator
(KiX Supporter)
2009-09-24 09:12 AM
Re: Removal of Duplicates in ListView

You haven't taken account of the effect of removing an item and reordering the list that I described in the earlier post.

You are now comparing only element 0, which is a step backwards from your original code.

You are no longer checking the unique field, which means you are going to delete entries where the unique key foes not match.