I was browsing the internet and trying things for quite a while. Finally, I managed to reach what I think you asked.
Using Xfwm4 on Linux Mint 17.3 Xfce, I ended up on the webpage http://docs.xfce.org/xfce/xfce4-panel/start which links to the page for "Windows Buttons" on http://docs.xfce.org/xfce/xfce4-panel/tasklist. On the screenshot shown there, you can see the option "Window grouping" where "Never" is chosen. If this is changed to "Always", two or more windows are grouped like we know it from e.g. MS Windows.
The question is now how to reach this menu. "Window Buttons" is the standard item in the panel to switch between different windows. The difficulty now is to right click on this item in the panel without right-clicking on a specific window. If the space is not enough, just navigate to the "Panel Preferences" and increase the "Number of Rows" until you find some space in the "Window Buttons" item. Then right-click this space and choose "Properties". A window like the screenshot in the link above pops up and you can change the desired option.