CODEWITHSUNDEEP
javaeclipseautomationsikulisikuli-x
Hydrochaeris
Hydrochaeris

Reputation: 21

Sikuli on Java does not recognize images

I have been trying to automatize some tasks on my computer and did choose Sikuli from Java to do so (I work with Java everyday and didn't know any automation tool using java, sikuli was the first I found). I use java with maven and eclipse as IDE. I have added Sikuli as a Maven dependency.

<dependency>
    <groupId>com.sikulix</groupId>
    <artifactId>sikulixapi</artifactId>
    <version>2.0.5</version>
</dependency>

I tried to do some simple stuff. I did a few screenshots of parts of my screen using windows' screenshot tool, and wanted sikuli to hover it. It works quite fine for one image, but not at all for the others. It seems that the bigger the image the better it works as I did not have success for any small images. The one working is a screen of a whole window (reduced to ~1/4 of my screen). I also tried to find a button inside this window, to find the windows logo on bottom left, to find a screen of my package explorer, but none work correctly.

I played with similar() using various values, but it didn't improve the results. In some cases (button inside the window) it did find a result for some low similar value, but it was another button. The weird part is : its finding this other button which is bright blue, while the one i'm looking for is purple.

My pc background never changes, I did some screen.highlight() and its looking at the correct screen (dual screen). It's not an issue with the path to images (already solved this one).

Do you have any idea of what I could try ? I have read about people having different success rate depending on whether they were using Sikuli IDE or another IDE. So maybe I could give sikuli IDE a try.

I can give code samples as soon as I am back home.

The code I'm using to test :

public class CleanTest {
    
    static Screen screen = new Screen();

    public static void main(String[] args) throws FindFailed, AWTException, IOException, InterruptedException {
        String pathYourSystem = System.getProperty("user.dir") + "\\";
        System.out.println(pathYourSystem);
        Pattern pLauncher = new Pattern(pathYourSystem+"img\\full_launcher.PNG").similar(0.9d);
        Desktop.getDesktop().open(new File("path_to_an_exe_opening_a_launcher"));
        screen.wait(pLauncher, 300);
        screen.mouseMove();
        System.out.println("launcher found");
    }

}

It works with the "full launcher" image, but it doesn't find a sub-part of the launcher (a button). I tried to make some code to test if there was some threshold for the similar parameter :

double similarValue = 1d;
        Pattern pLauncher = new Pattern(pathYourSystem+"img\\the_button.PNG").similar(similarValue);
        Desktop.getDesktop().open(new File("path_to_an_exe_opening_a_launcher"));
        
        while(!screen.has(pLauncher)) {
            similarValue-=0.1;
            pLauncher = new Pattern(pathYourSystem+"img\\login.PNG").similar(similarValue);
        }
        System.out.println(similarValue);
        screen.mouseMove();

it finds something at around 0.5, but it's a totally different button.

Thank you !

EDIT: if someone has the same issue, try to use sikulix IDE to take the screenshots. It works with the screenshots taken by the IDE.

Upvotes: 0

Views: 961

Answers (1)

RaiMan
RaiMan

Reputation: 1143

This is a simple test, that completely stays within the SikuliX features.

import org.sikuli.basics.Debug;
import org.sikuli.script.*;

public class SikulixTest {
  public static void main(String[] args) {
    System.out.println("SikulixTest");
    Screen scr = new Screen();
    // craete an image to be searched on the screen
    Image img = new Image(scr.userCapture());
    // try to find it
    Match mImg = scr.exists(img);
    if (mImg != null) {
      // show sthg. when found
      Debug.info("%s", mImg);
      mImg.highlight(2);
    }
  }
}

This is RaiMan from SikuliX

Upvotes: 1

Related Questions