Every now and then I come across some internal Matlab bugs. In many cases I find a workaround and move on, sometimes bothering to report the bugs to MathWorks support, but often not. In truth, it’s a bit frustrating to hear the standard response that the issue [or "unexpected behavior", but never "bug" - apparently that's a taboo word] “has been reported to the development team and they will consider fixing it in one of the future releases of MATLAB”.

To date I’ve reported dozens of bugs and as far as I can tell, few if any of them have actually been fixed, years after I’ve reported them. None of them appear on Matlab’s official bug parade, which is only a small subset of the full list that MathWorks keeps hidden for some unknown reason (update: see the discussion in the comments thread below, especially the input by Steve Eddins). Never mind, I don’t take it personally, I simply find a workaround and move on. I’ve already posted about this before. Today I’ll discuss two additional bugs I’ve run across once-too-often, and my workarounds:

• Saving non-Latin Command Window text using diary
• Printing GUIs reliably

Nothing really earth-shattering, but annoying nonetheless.

Saving non-Latin Command Window text using diary

The diary function is well-known for saving Matlab’s Command-Window (CW) text to a file. The function has existed for the past two decades at least, possibly even longer.

Unfortunately, perhaps the developer never thought that Matlab would be used outside the Americas and Western Europe, otherwise I cannot understand why to this day diary saves the text in ASCII format rather than the UTF-16 variant used by the CW. This works ok for basic Latin characters, but anyone who outputs Chinese, Japanese, Korean, Hindi, Arabic, Hebrew or other alphabets to the CW, and tries to save it using diary, will find the file unreadable.

Here is a sample illustrative script, that outputs the Arabic word salaam (peace, سلام) to the CW and then tries to save this using diary. If you try it, you will see it ok in the CW, but garbage text in the generated text file:

>> fname='diary_bug.txt'; diary(fname); disp(char([1587,1604,1575,1605])); diary off; winopen(fname)
سلام

The problem is that since diary assumes ASCII characters, any characters having a numeric value above 255 get truncated and are stored as invalid 1-byte characters, char(26) in this case.

Here’s my workaround:

% Output Command Window text to a text file
function saveCmdWinText(filename)
    cmdWinDoc = com.mathworks.mde.cmdwin.CmdWinDocument.getInstance;
    txt = char(cmdWinDoc.getText(0,cmdWinDoc.getLength));
    fid = fopen(filename,'W');
    fwrite(fid,txt,'uint16');  % store as 2-byte characters
    fclose(fid);
    %winopen(filename);  % in case you wish to verify...
end

This works well, saving the characters in their original 2-byte format, for those alphabets that use 2-bytes: non-basic Latins, Greek, Cyrillic, Armenian, Arabic, Hebrew, Coptic, Syriac and Tāna (I don’t think there are more than a handful of Matlab users who use Coptic, Syriac or Tāna but never mind). However, UTF-8 specifies that CJK characters need 3-4 bytes and this is apparently not supported in Matlab, whose basic char data type only has 2 bytes, so I assume that Chinese, Japanese and Korean will probably require a different solution (perhaps the internal implementation of char and the CW is different in the Chinese/Japanese versions of Matlab, I really don’t know. If this is indeed the case, then perhaps a variant of my workaround can also be used for CJK output).

Correction #1: I have learned since posting (see Steve Eddins’ comment below) that Matlab actually uses UTF-16 rather than UTF-8, solving the CJK issue. I humbly stand corrected.

Correction #2: The saveCmdWinText code above saves the CW text in UTF-16 format. This may be problematic in some text editors that are not UTF-savvy. For such editors (or if your editor get confused with the various BOM/endianness options), consider saving the data in UTF-8 format – again, assuming you’re not using an alphabet [such as CJK] outside the UTF-8 range (thanks Rob):

function saveCmdWinText_UTF8(filename)
    cmdWinDoc = com.mathworks.mde.cmdwin.CmdWinDocument.getInstance;
    txt = char(cmdWinDoc.getText(0,cmdWinDoc.getLength));
    fid = fopen(filename,'W','n','utf-8');
    fwrite(fid,txt,'char');
    fclose(fid);
    %winopen(filename);  % in case you wish to verify...
end

Also, this workaround is problematic in the sense that it’s a one-time operation that stores the entire CW text that is visible at that point. This is more limited than diary‘s ability to start and stop output recording in mid-run, and to record output on-the-fly (rather than only at the end). Still, it does provide a solution in case you output non-ASCII 2-byte characters to the CW.

Update: I plan to post a utility to the Matlab File Exchange in the near future that will mimic diary‘s ability to start/stop text recording, rather than simply dumping the entire CW contents to file. I’ll update here when this utility is ready for download.

There are various other bugs related to entering non-Latin (and specifically RTL) characters in the CW and the Matlab Editor. Solving the diary bug is certainly the least of these worries. Life goes on…

p.s. – I typically use this translator to convert from native script to UTF codes that can be used in Matlab. I’m sure there are plenty of other translators, but this one does the job well enough for me.

For people interested in learning more about the Command Window internals, take a look at my cprintf and setPrompt utilities.

cprintf usage examples

setPrompt usage examples

Printing GUIs reliably

Matlab has always tried to be far too sophisticated for its own good when printing figures. There’s plenty of internal code that tries to handle numerous circumstances in the figure contents for optimal output. Unfortunately, this code also has many bugs. Try printing even a slightly-complex GUI containing panels and/or Java controls and you’ll see components overlapping each other, not being printed, and/or being rendered incorrectly in the printed output. Not to mention the visible flicker that happens when Matlab modifies the figure in preparation for printing, and then modifies it back to the original.

All this when a simple printout of a screen-capture would be both much faster and 100% reliable.

Which is where my ScreenCapture utility comes in. Unlike Matlab’s print and getframe, ScreenCapture takes an actual screen-capture of an entire figure, or part of a figure (or even a desktop area outside any Matlab figure), and can then send the resulting image to a Matlab variable (2D RGB image), an image file, system clipboard, or the printer. We can easily modify the <Print> toolbar button and menu item to use this utility rather than the builtin print function:

Matlab's default toolbar Print action

hToolbar = findall(gcf,'tag','FigureToolBar');
hPrintButton = findall(hToolbar, 'tag','Standard.PrintFigure');
set(hPrintButton, 'ClickedCallback','screencapture(gcbf,[],''printer'')');
 
hPrintMenuItem = findall(gcf, 'type','uimenu', 'tag','printMenu');
set(hPrintMenuItem,      'Callback','screencapture(gcbf,[],''printer'')');

This prints the entire figure, including the frame, menubar and toolbar (if any). If you just wish to print the figure’s content area, then make sure to create a top-level uipanel that spans the entire content area and in which all the contents are included. Then simply pass this top-level container handle to ScreenCapture:

hTopLevelContainer = uipanel('BorderType','none', 'Parent',gcf, 'Units','norm', 'Pos',[0,0,1,1]);
...
hToolbar = findall(gcf,'tag','FigureToolBar');
hPrintButton = findall(hToolbar, 'tag','Standard.PrintFigure');
set(hPrintButton, 'ClickedCallback',@(h,e)screencapture(hTopLevelContainer,[],'printer'));
 
hPrintMenuItem = findall(gcf, 'type','uimenu', 'tag','printMenu');
set(hPrintMenuItem,      'Callback',@(h,e)screencapture(hTopLevelContainer,[],'printer'));

In certain cases (depending on platform/OS/Matlab-release), the result may capture a few pixels from the figure’s window frame. This can easily be corrected by specifying a small offset to ScreenCapture:

set(hPrintButton, 'ClickedCallback',@(h,e)printPanel(hTopLevelContainer));
set(hPrintMenuItem,      'Callback',@(h,e)printPanel(hTopLevelContainer));
 
function printPanel(hTopLevelContainer)
    pos = getpixelposition(hTopLevelContainer);
    screencapture(hTopLevelContainer, pos+[2,4,0,0], 'printer');
end

1. A couple of internal Matlab bugs and workarounds A couple of undocumented Matlab bugs have simple workarounds. ...
2. setPrompt – Setting the Matlab Desktop prompt The Matlab Desktop's Command-Window prompt can easily be modified using some undocumented features...
3. Images in Matlab uicontrols & labels Images can be added to Matlab controls and labels in a variety of manners, documented and undocumented. ...
4. Types of undocumented Matlab aspects This article lists the different types of undocumented/unsupported/hidden aspects in Matlab...

All Matlab figures have a standard frame around them, consisting of a border and title bar. In some cases it could be useful to present a figure window, displaying only the contents without the border and title bar. Such a borderless (undecorated) window is not possible in Matlab. Well, at least not in supported/documented Matlab…

Readers of this blog and/or my Matlab-Java programming book are aware that Matlab’s GUI is based on Java Swing. In the end, every Matlab figure window is a simple Java JFrame, and JFrames have a setUndecorated(true) method that can be called to remove the border and title bar.

An undecorated Matlab figure window

Attempt #1 – direct invocation

Unfortunately, we cannot directly call setUndecorated in Matlab, because it only works when the JFrame has not yet been displayed:

% Get the figure's underlying Java JFrame reference handle
>> mjf = get(handle(gcf), 'JavaFrame');
>> jWindow = mjf.fHG2Client.getWindow  % or: mjf.getAxisComponent.getTopLevelAncestor
jWindow =
com.mathworks.hg.peer.FigureFrameProxy$FigureFrame[fClientProxyFrame,740,-761,576x509,...]
 
% Try to remove decoration
>> jWindow.setUndecorated(true)
Java exception occurred:
java.awt.IllegalComponentStateException: The frame is displayable.
	at java.awt.Frame.setUndecorated(Unknown Source)

On the other hand, if we try to call setUndecorated on a new invisible figure then we’d see that the JFrame component is not yet created by Matlab:

% Create a simple Matlab figure
hFig = figure('Name','Plot example', 'Visible','off');
 
% Get the underlying Java JFrame reference handle (empty)
>> mjf = get(handle(hFig),'JavaFrame');
>> jWindow = mjf.getAxisComponent.getTopLevelAncestor
jWindow =
     []
>> jWindow = mjf.fHG2Client.getWindow
jWindow =
     []
 
>> jWindow.setUndecorated(true)
Attempt to reference field of non-structure array.

So we have a catch-22 situation: we can’t call setUndecorated until the JFrame is created, but Matlab only creates it when it displays the figure window, and then it’s too late to undecorate…

Attempt #2 – reparenting: an optical illusion

One way that I found around this problem is to reparent the Matlab JFrame‘s content onto a pure Java JFrame that has been made undecorated. To hide the Matlab figure after creating the JFrame, we can simply move the figure’s position to be outside the visible monitor area:

% Create a simple Matlab figure (visible, but outside monitor area)
t = 0 : 0.01 : 10;
hFig = figure('Name','Plot example', 'ToolBar','none', 'MenuBar','none');
hLine = plot(t, cos(t));
hButton = uicontrol('String','Close', 'Position',[307,0,45,16]);
 
% Ensure that everything is rendered, otherwise the following will fail
drawnow;
 
% Get the underlying Java JFrame reference handle
mjf = get(handle(hFig), 'JavaFrame');
jWindow = mjf.fHG2Client.getWindow;  % or: mjf.getAxisComponent.getTopLevelAncestor
 
% Get the content pane's handle
mjc = jWindow.getContentPane;
mjr = jWindow.getRootPane;  % used for the offset below
 
% Create a new pure-Java undecorated JFrame
figTitle = jWindow.getTitle;
jFrame = javaObjectEDT(javax.swing.JFrame(figTitle));
jFrame.setUndecorated(true);
 
% Move the JFrame's on-screen location just on top of the original
jFrame.setLocation(mjc.getLocationOnScreen);
 
% Set the JFrame's size to the Matlab figure's content size
%jFrame.setSize(mjc.getSize);  % slightly incorrect by root-pane's offset
jFrame.setSize(mjc.getWidth+mjr.getX, mjc.getHeight+mjr.getY);
 
% Reparent (move) the contents from the Matlab JFrame to the new JFrame
jFrame.setContentPane(mjc);
 
% Make the new JFrame visible
jFrame.setVisible(true);
 
% Hide the Matlab figure by moving it off-screen
pos = get(hFig,'Position');
set(hFig, 'Position',[-1000,-1000,pos(3:4)]);
drawnow;

Matlab figures are not pure JFrame but rather a subclass (com.mathworks.widgets.desk.DTSingleClientFrame). So instead of creating a new JFrame, we could create a new instance of Matlab’s class instead, potentially solving a few problems:

jDesktop = com.mathworks.mde.desk.MLDesktop.getInstance;
jFrame = javaObjectEDT(com.mathworks.widgets.desk.DTSingleClientFrame(jDesktop, figTitle));
% ...the rest is exactly the same as above...

Either way, we now get a nice undecorated window containing our Matlab contents (see screenshot above).

Once the undecorated JFrame becomes visible, we should not hide or delete the original Matlab figure, because this will stop rendering of the contents.

Working with the undecorated frame

We can modify the Matlab figure and its contents normally, just as if they still appeared within the original Matlab figure. This can be used to display a dynamic application splash-screen, that displays information on various initialization/loading steps. Or alternately we could use it to display a system health-monitor (small panel with red/green indicators), or maybe a graph with streaming market data or live news alerts. The usages are endless, limited only by your imagination, not by Matlab.

For example, let’s modify our plot dynamically using a timer, and set the close button’s callback to dismiss the new JFrame and original figure:

% Set a timer to dynamically update the plot every 0.1 sec
start(timer('TimerFcn', {@timerCallback,hLine}, 'ExecutionMode','fixedRate', 'Period',0.1));
 
% Set the close button callback
set(hButton, 'Callback',{@closeCallback,jFrame});
 
% This is the close button's callback
function closeCallback(hButton, eventData, jFrame)
   delete(ancestor(hButton,'figure'));
   dispose(jFrame);
end
 
% This is the plot timer's callback
function timerCallback(hTimer, eventData, hLine)
   xdata = get(hLine,'XData') + 0.1;
   set(hLine, 'XData',xdata, 'YData',cos(xdata));
   xlim([min(xdata) max(xdata)]);
end

Note: if you don’t hide the toolbar/menubar in the original Matlab figure, some of their functions will not work properly in the new JFrame (e.g., zoom, pan etc.). But in most use-cases we do not want a toolbar/menubar in an undecorated window. The example above showed an undecorated window without the toolbar/menubar.

If we wish to avoid having the new JFrame appear in the Operating System’s taskbar, we can use the following command:

jFrame.setType(javaMethod('valueOf','java.awt.Window$Type','UTILITY'))

Note that this command must be executed before the JFrame is made visible. Also note that it only works with Java 7, in other words Matlab R2013b (8.2) or newer (or if you are very adventurous and happen to use an older Matlab with a custom Java 7 installation).

But now that there is no taskbar component, how can we transfer focus to our new undecorated window? In other words, if another windows hides our new undecorated frame, how can we get it back on top?

A simple solution is to set the Matlab figure frame’s FocusGainedCallback to requestFocus for the newly created jFrame. Then, when we click the Matlab figure’s button on the taskbar, the new jFrame will pop into focus. In effect, this provides the optical illusion that the taskbar button refers to the undecorated window (since the actual Matlab figure is positioned beyond the monitor’s area):

hjWindow = handle(jWindow, 'CallbackProperties');
set(hjWindow, 'FocusGainedCallback', @(h,e)jFrame.requestFocus);

Likewise, we should instrument our Matlab figure so that when it is closed/deleted, so too is our new jFrame:

set(hjWindow, 'WindowClosedCallback', @(h,e)jFrame_.dispose);

undecorateFig and redecorateFig

I have encapsulated all of the above in a couple of very easy-to-use utilities that I just posted on the Matlab File Exchange: undecorateFig, redecorateFig. Using them can’t get any simpler than this:

undecorateFig;        % undecorate the current figure (gcf)
undecorateFig(hFig);  % hFig is any GUI handle (needs to be visible)
 
redecorateFig;        % redecorate the current figure (gcf)
redecorateFig(hFig);  % hFig is any GUI handle

Any sufficiently advanced technology is indistinguishable from magic – Arthur C. Clarke  

Have you made some interesting use of this nice feature in your application? If so, please share it in a comment below.

A suggested related mini project for anyone interested: add an image reflection of the current figure contents beneath the figure. You could use my ScreenCapture utility or directly use java.awt.Robot.createScreenCapture(mjc.getLocationOnScreen), process the resulting image (blur, apply transparency gradient, crop at 50% height etc.) and place the resulting image within an undecorated JFrame placed directly beneath the main figure. You can instrument the figure (hjWindow)’s ComponentMovedCallback and ComponentResizedCallback to move/resize the reflection JFrame whenever the parent figure moves or is resized. You could use a timer to periodically update the reflection image so that it remains synchronized with the parent. You’ll probably also want to add a nice toggle-button to the figure’s toolbar to turn the reflection on/off. Maybe I’ll hack this project someday when I have some spare time. Or maybe someone will beat me to it… Care to try?

1. Minimize/maximize figure window Matlab figure windows can easily be maximized, minimized and restored using a bit of undocumented magic powder...
2. Transparent Matlab figure window Matlab figure windows can be made fully or partially transparent/translucent or blurred - this article explains how...
3. Blurred Matlab figure window Matlab figure windows can be blurred using a semi-transparent overlaid window - this article explains how...
4. Customizing figure toolbar background Setting the figure toolbar's background color can easily be done using just a tiny bit of Java magic powder. This article explains how. ...

As a followup to my recent post on capturing and using HG2′s new graphic events, I would like to welcome back guest blogger Robert Cumming, who developed a commercial Matlab GUI framework. Today, Robert will highlight an unusual use of event listeners to enable print output customizations by tapping into print-related events.

One of my toolbox clients wanted to ensure that every print-out, of any Matlab GUI, would contain a datestamp footer and a project id header.

Initially I thought this would be a simple task, since the GUI framework already has saveAs and copyToClipboard methods that users could utilize. Moreover, as Yair has shown some years ago, we can easily adapt the standard callbacks associated with the Matlab figure toolbar’s <Print> button and the menu-bar’s Print action. We can even customize the standard print setup. However, all this does not help when directly invoking printout commands from the Matlab command line or from within the user program, for example:

% Create the GUI
[x,y,z] = peaks(75); 
surf(x,y,z);
 
% printout via print() or export_fig
print(gcf, .... )
export_fig filename  % see http://undocumentedmatlab.com/blog/export_fig

Printout with the expected header and footer stamps

This initially posed a bit of a challenge – how could I capture the print event?

The key to solving this was using events in a different way. Rather than listen for a print event, we listen for something that print interacts with in the figure window. This will solve the problem for both print and export_fig, since export_fig uses print internally.

We can listen to get and set events on many Matlab properties. In this instance we can trap get events on a figure’s PaperPositioMode, or PaperSize property. print, saveas and export_fig all access these properties before doing the actual printing, so by trapping the event we can run our own function before the printing is done. For example:

classdef PrintListenerDemo
  properties
    referenceNo = 'R1234-56'
  end
 
  methods (Access=public)
    function obj = PrintListenerDemo
      hFig = figure;
      [x,y,z] = peaks(75);
      surf ( x, y, z );
 
      % add listener on event to prepare the figure for printing
      addlistener ( hFig, 'PaperPositionMode', 'PreGet', @obj.Prep );
 
      % Testing only:
      evalin ( 'base', 'print ( gcf, ''-dmeta'' )' );  % simulate user action in command-line
      print(hFig,'-dmeta');    % simulate programmatic printout
    end
 
    function Prep(obj, hFig, varargin)
      disp ( 'running function in preparation for printing.' );
 
      % Add datestamp and reference id UIControls
      str = sprintf ( 'Printed @ %s', datestr(now) );
      uicontrol ( 'parent',hFig, 'style','text', 'units','normalized', ...
                  'position',[0 0 1 0.07], 'string',str, 'BackgroundColor','white' );
    end
  end
end

When we run the above we can see that the disp message is displayed twice, because the property has two get calls in the print function. We can easily solve this by saving a temp variable that is created the first time it is run.

The next thing we need to do is to clean up afterwards, i.e. remove the uicontrols and reset the temp variable to ensure that any future print events are captured.
Here you might think the second get call I mentioned above was post printing, but it’s not. So we need to capture another event to do the cleaning up.

In this instance I thought about what was likely to happen – some sort of user interaction, e.g. a mouse move, press or key press. So we can listen for any of those events and run our clean up method. This is shown in the full code below:

% PrintListenerDemo - Demo of print listener methodology
classdef PrintListenerDemo
  properties
    referenceNo = 'R1234-56'
  end
 
  methods (Access=public)
    function obj = PrintListenerDemo
      hFig = figure;
      [x,y,z] = peaks(75);
      surf ( x, y, z );
 
      % add listener on event to prepare the figure for printing
      addlistener ( hFig, 'PaperPositionMode', 'PreGet', @obj.Prep );      
 
      % add a few different ways to clean up
      addlistener ( hFig, 'ButtonDown', @obj.CleanUp );
      addlistener ( hFig, 'WindowKeyPress', @obj.CleanUp );
      addlistener ( hFig, 'WindowMouseMotion', @obj.CleanUp );
 
      % Testing only:
      evalin ( 'base', 'print ( gcf, ''-dmeta'' )' );  % simulate user action in command-line
      print(hFig,'-dmeta');    % simulate programmatic printout
    end
 
    function Prep(obj, hFig, varargin)
      try
        hFig.ApplicationData.printPrepDone;
        return % If variable set do not run this method again
      end
      disp ( 'preparing for print.' );
 
      % create the date string
      str1 = sprintf ( 'Printed @ %s', datestr(now) );
      str2 = sprintf ( 'Reference: %s', obj.referenceNo );
 
      % create 2 UI controls to contain the information requested in the image
      hFig.ApplicationData.UIC1 = uicontrol ( 'parent',hFig, 'style','text', 'units','norm', 'string',str1, 'BackgroundColor','w', 'position',[0 0 1 0.07] );
      hFig.ApplicationData.UIC2 = uicontrol ( 'parent',hFig, 'style','text', 'units','norm', 'string',str2, 'BackgroundColor','w', 'position',[0 0.93 1 0.07] );
 
      % Save a temp variable to stop this function being called many times.
      hFig.ApplicationData.printPrepDone = true;
    end
 
    function CleanUp(obj, hFig, varargin)
      try
        if hFig.ApplicationData.printPrepDone
          % Clean up the uicontrols
          disp ( 'clean up post printing event' );
          delete ( hFig.ApplicationData.UIC1 );
          delete ( hFig.ApplicationData.UIC2 );
 
          % remove the temp variable, so the next print job will be captured
          hFig.ApplicationData = rmfield ( hFig.ApplicationData, 'printPrepDone' );
        end
      end
    end
  end
end

It is noted here that the datestamp and the reference number remain visible until the user interacts with the GUI but that’s a small price to pay.

Any suggestions on other ways to clean up the print event afterwards? If so, please post a comment below. Alternatively if you have used events and listeners in an unusual way please share this as well.

Conclusions

1. It is possible to capture printout events in your code even when the print command is invoked programmatically, without a direct reference to your code
2. We can use events for things that we might never have initially considered

1. Customizing print setup Matlab figures print-setup can be customized to automatically prepare the figure for printing in a specific configuration...
2. UDD Events and Listeners UDD event listeners can be used to listen to property value changes and other important events of Matlab objects...
3. Detecting window focus events Matlab does not have any documented method to detect window focus events (gain/loss). This article describes an undocumented way to detect such events....
4. Matlab callbacks for Java events Events raised in Java code can be caught and handled in Matlab callback functions - this article explains how...

An annoying side-effect of Matlab figure uimodes (zoom, pan and rotate3d) is that they disable the user’s figure-wide callbacks (KeyPressFcn, KeyReleaseFcn, WindowKeyPressFcn, WindowKeyReleaseFcn, WindowButtonUpFcn, WindowButtonDownFcn and WindowScrollWheelFcn). In most cases, users will indeed expect the mouse and keyboard behavior to change when these modes are active (selecting a zoom region with the mouse for instance). However, in certain cases we may need to retain our custom callback behavior even when the modes are active. This is particularly relevant for the keyboard callbacks, which are not typically used to control the modes and yet may be very important for our figure’s interactive functionality. Unfortunately, Matlab’s mode manager installs property listeners that prevent users from modifying these callback when any mode is active:

>> hFig=figure; plot(1:5); zoom on
>> set(hFig, 'KeyPressFcn', @myKeyPressCallback)
Warning: Setting the "KeyPressFcn" property is not permitted while this mode is active.
(Type "warning off MATLAB:modes:mode:InvalidPropertySet" to suppress this warning.)
 
> In matlab.uitools.internal.uimodemanager>localModeWarn (line 211)
  In matlab.uitools.internaluimodemanager>@(obj,evd)(localModeWarn(obj,evd,hThis)) (line 86)
 
>> get(hFig, 'KeyPressFcn')  % the KeyPressFcn callback set by the mode manager
ans =
    @localModeKeyPressFcn
    [1x1 matlab.uitools.internal.uimode]
    {1x2 cell                          }

The question of how to override this limitation has appeared multiple times over the years in the CSSM newsgroup (example1, example2) and the Answers forum, most recently yesterday, so I decided to dedicate today’s article to this issue.

In Matlab GUI, a “mode” (aka uimode) is a set of defined behaviors that may be set for any figure window and activated/deactivated via the figure’s menu, toolbar or programmatically. Examples of predefined modes are plot edit, zoom, pan, rotate3d and data-cursor. Only one mode can be active in a figure at any one time – this is managed via a figure’s ModeManager. Activating a figure mode automatically deactivates any other active mode, as can be seen when switching between plot edit, zoom, pan, rotate3d and data-cursor modes.

This mode manager is implemented in the uimodemanager class in %matlabroot%/toolbox/matlab/uitools/+matlab/+uitools/internal/@uimodemanager/uimodemanager.m and its sibling functions in the same folder. Until recently it used to be implemented in Matlab’s old class system, but was recently converted to the new MCOS (classdef) format, without an apparent major overhaul of the underlying logic (which I think is a pity, but refactoring naturally has lower priority than fixing bugs or adding new functionality).

Anyway, until HG2 (R2014b), the solution for bypassing the mode-manager’s callbacks hijacking was as follows (credit: Walter Roberson):

% This will only work in HG1 (R2014a or earlier)
hManager = uigetmodemanager(hFig);
set(hManager.WindowListenerHandles, 'Enable', 'off');
set(hFig, 'WindowKeyPressFcn', []);
set(hFig, 'KeyPressFcn', @myKeyPressCallback);

In HG2 (R2014b), the interface of WindowListenerHandles changed and the above code no longer works:

>> set(hManager.WindowListenerHandles, 'Enable', 'off');
Error using set
Cannot find 'set' method for event.proplistener class.

Luckily, in MathWorks’ refactoring to MCOS, the property name WindowListenerHandles was not changed, nor its status as a public read-only hidden property. So we can still access it directly. The only thing that changed is its meta-properties, and this is due not to any change in the mode-manager’s code, but rather to a much more general change in the implementation of the event.proplistener class:

>> hManager.WindowListenerHandles(1)
ans =
  proplistener with properties:
 
       Object: {[1x1 Figure]}
       Source: {7x1 cell}
    EventName: 'PreSet'
     Callback: @(obj,evd)(localModeWarn(obj,evd,hThis))
      Enabled: 0
    Recursive: 0

In the new proplistener, the Enabled meta-property is now a boolean (logical) value rather than a string, and was renamed Enabled in place of the original Enable. Also, the new proplistener doesn’t inherit hgsetget, so it does not have set and get methods, which is why we got an error above; instead, we should use direct dot-notation.

In summary, the code that should now be used, and is backward compatible with old HG1 releases as well as new HG2 releases, is as follows:

% This should work in both HG1 and HG2:
hManager = uigetmodemanager(hFig);
try
    set(hManager.WindowListenerHandles, 'Enable', 'off');  % HG1
catch
    [hManager.WindowListenerHandles.Enabled] = deal(false);  % HG2
end
set(hFig, 'WindowKeyPressFcn', []);
set(hFig, 'KeyPressFcn', @myKeyPressCallback);

During an active mode, the mode-manager disables user-configured mouse and keyboard callbacks, in order to prevent interference with the mode’s functionality. For example, mouse clicking during zoom mode has a very specific meaning, and user-configured callbacks might interfere with it. Understanding this and taking the proper precautions (for example: chaining the default mode’s callback at the beginning or end of our own callback), we can override this default behavior, as shown above.

Note that the entire mechanism of mode-manager (and the related scribe objects) is undocumented and might change without notice in future Matlab releases. We were fortunate in the current case that the change was small enough that a simple workaround could be found, but this may possibly not be the case next time around. It’s impossible to guess when such features will eventually break: it might happen in the very next release, or 20 years in the future. Since there are no real alternatives, we have little choice other than to use these features, and document the risk (the fact that they use undocumented aspects). In your documentation/comments, add a link back here so that if something ever breaks you’d be able to check if I posted any fix or workaround.

For the sake of completeness, here is a listing of the accessible properties (regular and hidden) of both the mode-manager as well as the zoom uimode, in R2015b:

>> warning('off', 'MATLAB:structOnObject');  % suppress warning about using structs (yes we know it's not good for us...)
>> struct(hManager)
ans = 
                    CurrentMode: [1x1 matlab.uitools.internal.uimode]
                  DefaultUIMode: ''
                       Blocking: 0
                   FigureHandle: [1x1 Figure]
          WindowListenerHandles: [1x2 event.proplistener]
    WindowMotionListenerHandles: [1x2 event.proplistener]
                 DeleteListener: [1x1 event.listener]
            PreviousWindowState: []
                RegisteredModes: [1x1 matlab.uitools.internal.uimode]
 
>> struct(hManager.CurrentMode)
ans = 
    WindowButtonMotionFcnInterrupt: 0
                UIControlInterrupt: 0
                   ShowContextMenu: 1
                         IsOneShot: 0
                    UseContextMenu: 'on'
                          Blocking: 0
               WindowJavaListeners: []
                    DeleteListener: [1x1 event.listener]
              FigureDeleteListener: [1x1 event.listener]
                    BusyActivating: 0
                              Name: 'Exploration.Zoom'
                      FigureHandle: [1x1 Figure]
             WindowListenerHandles: [1x2 event.proplistener]
                   RegisteredModes: []
                       CurrentMode: []
               ModeListenerHandles: []
               WindowButtonDownFcn: {@localWindowButtonDownFcn  [1x1 matlab.uitools.internal.uimode]}
                 WindowButtonUpFcn: []
             WindowButtonMotionFcn: {@localMotionFcn  [1x1 matlab.uitools.internal.uimode]}
                 WindowKeyPressFcn: []
               WindowKeyReleaseFcn: []
              WindowScrollWheelFcn: {@localButtonWheelFcn  [1x1 matlab.uitools.internal.uimode]}
                     KeyReleaseFcn: []
                       KeyPressFcn: {@localKeyPressFcn  [1x1 matlab.uitools.internal.uimode]}
                      ModeStartFcn: {@localStartZoom  [1x1 matlab.uitools.internal.uimode]}
                       ModeStopFcn: {@localStopZoom  [1x1 matlab.uitools.internal.uimode]}
                  ButtonDownFilter: []
                     UIContextMenu: []
                     ModeStateData: [1x1 struct]
                WindowFocusLostFcn: []
          UIControlSuspendListener: [1x1 event.listener]
      UIControlSuspendJavaListener: [1x1 handle.listener]
                   UserButtonUpFcn: []
               PreviousWindowState: []
                 ActionPreCallback: []
                ActionPostCallback: []
           WindowMotionFcnListener: [1x1 event.listener]
                       FigureState: [1x1 struct]
                        ParentMode: []
                     DefaultUIMode: ''
             CanvasContainerHandle: []
                            Enable: 'on'

1. Matlab callbacks for Java events Events raised in Java code can be caught and handled in Matlab callback functions - this article explains how...
2. uisplittool & uitogglesplittool callbacks Matlab's undocumented uisplittool and uitogglesplittool are powerful toolbar controls - this article explains how to customize their behavior...
3. Determining axes zoom state The information of whether or not an axes is zoomed or panned can easily be inferred from an internal undocumented object....
4. Removing user preferences from deployed apps An unsupported MathWorks Technical Solution explains how to remove private information from deployed (compiled) matlab applications. ...

Matlab figures have a documented property called SelectionType that returns information about keypress modifiers such as or that were pressed during mouse clicks. Using this property has several drawbacks IMHO:

• The reported SelectionType value is 'extend' for shift-clicks and 'alt' for ctrl-clicks, not very intuitive.
• There is no support for alt-clicks, which are reported as regular ('normal') mouse clicks. In fact, 'alt' is reported for ctrl-clicks rather than for alt-clicks, which is very confusing.
• There is no support for modifier combinations such as ctrl+shift-click. These again are reported as regular ('normal') mouse clicks.
• SelectionType is only updated for mouse clicks, not for keyboard clicks. This again is very confusing. To extract the keypress modifier for key-click events we need to get the Modifier property of the key-press event, and this returns a cell-array of strings (e.g., {'shift','control','alt'}). Note that in this case, unlike SelectionType, the modifier names are as expected, alt-clicks is recognised and so are modifier combinations.

% Documented: we need to get the modifier data in two different manners
set(gcf, 'WindowButtonDownFcn', @(h,e) disp(get(gcf,'SelectionType')));  % mouse clicks: displays a single string: 'normal','alt','extend' or 'open'
set(gcf, 'WindowKeyPressFcn',   @(h,e) disp(e.Modifier));  % keyboard clicks: displays a cell-array of strings, e.g. {'shift','control','alt'}

The inconsistency between the functionality for mouse and keyboard clicks, and the limitations of the SelectionType property, are striking and most annoying. Some might say that it’s been like this for the past 2 decades so Matlab users have probably gotten used to it by now. But I must admit that after over 2 decades with Matlab I still need to refer to the documentation to figure out the correct behavior. Maybe that’s because I’m getting old, or maybe it means that the behavior is indeed not as intuitive as one could hope for.

For this reason, I was very happy to discover several years ago that there was a much simpler, easier and consistent solution, although (alas) undocumented. The trick is to simply query the undocumented hidden figure property CurrentModifier: CurrentModifier is updated during both mouse and keyboard clicks, in the same consistent manner, in both cases returning the same cell-array of strings as the Modifier property of the standard key-press event:

% Undocumented: the following displays a cell-array of strings having a consistent format, e.g. {'shift','control','alt'}
set(gcf, 'WindowButtonDownFcn', @(h,e) disp(get(gcf,'CurrentModifier')));  % mouse clicks
set(gcf, 'WindowKeyPressFcn',   @(h,e) disp(get(gcf,'CurrentModifier')));  % keyboard clicks

Checking whether any modifier was pressed becomes trivial:

modifiers = get(gcf,'CurrentModifier');
wasShiftPressed = ismember('shift',   modifiers);  % true/false
wasCtrlPressed  = ismember('control', modifiers);  % true/false
wasAltPressed   = ismember('alt',     modifiers);  % true/false

Hurrah for simplicity and consistency!

Note that despite the fact that CurrentModifier is hidden and undocumented, it has existed as-is for many years, and even survived (unchanged) the major transition from HG1 to HG2 in R2014b. This has to mean that MathWorks recognized the importance of CurrentModifier and took the deliberate decision (and associate dev effort) to preserve it. So why wasn’t this useful property made documented ages ago, or at the very least at the HG2 transition point? I don’t have a good answer to this riddle.

1. Minimize/maximize figure window Matlab figure windows can easily be maximized, minimized and restored using a bit of undocumented magic powder...
2. Customizing figure toolbar background Setting the figure toolbar's background color can easily be done using just a tiny bit of Java magic powder. This article explains how. ...
3. Blurred Matlab figure window Matlab figure windows can be blurred using a semi-transparent overlaid window - this article explains how...
4. uicontrol side-effect: removing figure toolbar Matlab's built-in uicontrol function has a side-effect of removing the figure toolbar. This was undocumented until lately. This article describes the side-effect behavior and how to fix it....

I would like to introduce Daniel Dolan of Sandia National Laboratories. Dan works on a variety of data analysis projects in Matlab, and is an active lurker on MATLAB Central. Dan has a habit of finding interesting bugs for the Mac version of Matlab. Today he will discuss graphic sizing in Matlab and important changes that occurred in release R2015b.

Matlab-generated graphics are often not displayed at their requested size. This problem has been known for some time and has a well-known solution: setting the root object’s ScreenPixelsPerInch property to the display’s actual DPI (dots per inch) value. Release R2015b no longer supports this solution, creating problems for publication graphics and general readability.

Physical sizing in R2015a vs. R2015b (click for full-size)

Physical sizing

Matlab supports graphic sizing in various physical units: inches, centimeters, and points. For example:

figure; axes('Box','on', 'Units','inches','Position',[0.3 0.3 4 4]);

requests to display an axes having square sizes measuring exactly 4″ (101.6 mm) each. It is evident, however, that the displayed axes is smaller than 4″. The mismatch between requested and physical size depends on the display and operating system — go ahead, try it on your system. The problem is particularly severe on Mac laptops, presumably even worse for those with Retina displays.

The problem is that Matlab cannot determine pixel size, which varies from one display to the other. Generating a figure spanning a particular number of pixels (e.g., 1024 x 768) is easy, but absolute physical units requires a conversion factor called ScreenPixelsPerInch, which is a root property (see related post on setting/getting default graphics property values):

DPI = 110;                             % dots per inch for my 27" Apple Cinema Display
set(0,    'ScreenPixelsPerInch',DPI);  % all releases prior to R2015b
set(groot,'ScreenPixelsPerInch',DPI);  % R2014b through R2015a

DPI values tend to be higher for laptops, usually in the 120-130 range. Retina displays are supposed to be >300 DPI, but I have not been able to test that myself.

There are several ways to determine the correct DPI setting for a particular display. It may be available in the hardware specifications, and it can be calculated from the diagonal size and the number of pixels. Unfortunately these methods are not always reliable. If you really care about physical sizing, the best approach is to actually calibrate your display. There are tools for doing this at Matlab Central, but it’s not hard to do manually:

• Create a figure.
• Manually resize the figure to match a convenient width. I often use a piece of US letter paper as 8.5″ guide on the display.
• Determine the width of the figure in pixels:

  set(gcf,'Units','pixels');
  pos = get(gcf,'Position');
  width = 8.5; % inches
  DPI = pos(3) / width;

I usually apply the DPI settings in my startup file so that Matlab begins with a calibrated display.

What changed in 2015b?

ScreenPixelsPerInch is a read-only property in R2015b, so display calibration no longer works. The following sequence of commands:

figure('Units','inches', 'PaperPositionMode','auto', 'Position',[0 0 4 4]);
set(gcf, 'MenuBar','none', 'ToolBar','none', 'DockControls','off', 'NumberTitle','off');
axes('FontUnits','points', 'FontSize',10);
image

now renders differently in R2015b than does for a calibrated display in R2015a. Differences between the two outputs are shown in the screenshot at the top of this post. The grid behind the figures was rendered at 8.5″ x 8.5″ inches on my display; if your browser’s zoom level isn’t 100%, it may appear larger or smaller.

A side effect of improper graphic sizing is that text is difficult to read — the uncalibrated axes labels are clearly smaller than 10 points. These examples were rendered on ~110 DPI display. Matlab assumes that Macs use 72 DPI (96 DPI on Windows), so graphics appear at 65% of the request size.

The loss of ScreenPixelsPerInch as an adjustable setting strongly affects anyone using Matlab for publication graphics. Scientific and engineering journals are extremly strict about figure widths. With a calibrated screen, figure appear exactly as they will when printed to a file (usually EPS or PDF). Figures are often made as small as possible to and densely packed to save journal space, and accurate sized display helps the author determine legibility. Displaying accurately sized graphics is very difficult in R2015b, which is unfortunate given the many enhancements in this release.

Developers who create graphical interfaces for other users should also care about this change. A common complaint I get is that text and control labels is too small to easily read. Screen calibration deals with this problem, but this option is no longer available.

Where do we go from here?

I reported the above issues to the Mathworks several months ago. It does not appear as a formal bug, but technical support is aware of the problem. The change is part of the “DPI aware” nature of release R2015b. So far I have found no evidence this release is any more aware of pixel size than previous releases, but my experience is limited to non-Retina Macs. I welcome input from users on other operating systems, particularly those with high-resolution displays.

To be fair, correct physical sizing is not an easy across the many platforms that Matlab runs on. Display resolution is particularly tricky when it changes during a Matlab session, such as when computer is connector to projector/television or a laptop is connected to a docking station.

Thankfully, printed graphic sizes are rendered correctly when a figure’s PaperPositionMode property is 'auto'. Many users can (and will) ignore the display problem if they aren’t dealing with strict size requirements and text legibility isn’t too bad. Some users may be willing to periodically print publication figures to externally verify sizing, but this breaks the interactive nature of Matlab figures.

A potential work around is the creating of a new figure class that oversizes figures (as needed) to account for a particular display. I started working on such a class, but the problem is more complicated than one might think:

• Child objects (axes, uicontrols, etc.) also must be resized if they are based on physical units.
• Resized objects must be temporarily restored to their original size for printing, and new objects must be tracked whenever they are added.
• Figure resolution may need to be changed when moving to different computer systems.

These capabilities are quite possible to implement, but this is a complicated solution to problem that was once easy to fix.

Retina displays don’t suffer as badly as one might think from the DPI mismatch. Even though the display specification may be greater than 200 DPI, OS X and/or Matlab must perform some intermediate size transformations. The effective DPI in R2015a is 110-120 for 13-15″ MacBook Pro laptops (at the default resolution). Objected sized with physical units still appear smaller than they should (~72/110), but not as small as I expected (<72/200).Effect pixel size can also be changed by switching between different monitor scalings. This isn't entirely surprising, but it can lead to some interesting results because Matlab only reads these settings at startup. Changing the display scaling during a session can cause square figures to appear rectangular. Also, the effective DPI changes for setting: I could reach values of ~60-110 DPI on an Apple Cinema Display.So where does this leave us? Display calibration was always a finicky matter, but at least in principle one could make graphics appear exactly the same size on two different displays. Now it seems that sizing is completely variable between operation systems, displays, and display settings. For publication graphics, there will almost always be a disconnect between figure size on the screen and the printed output; some iteration may be needed to ensure everything looks right in the finished output. For graphical interfaces, font sizes may need to generated in normalized units and then converted to pixels (to avoid resizing).Physical accuracy may not be important for non-publication figures, but the issue of text legibility remains. Some text objects--such as axes and tick labels--can easily be resized because the parent axes automatically adjusts itself as needed. Free floating text objects and uincontrols are much more difficult to deal with. Controls are often sized around the extent of their text label, so changing font sizes may require changes to the control position; adjacent controls may overlap after resizing for text clarity. Normalized units partially solve this problem, but their effect on uicontrols is not always desirable: do you really want push buttons to get larger/smaller when the figure is resized?Can you think of a better workaround to this problem? If so, then please post a comment below. I will be very happy to hear your ideas, as I'm sure others who have high resolution displays would as well.(cross-reference: CSSM newsgroup post)

Last week I wrote about my upcoming presentations in Tel Aviv and Munich, where I will discuss a Matlab-based financial application that uses some advanced GUI concepts. In today’s post I will review one of these concepts that could be useful in a wide range of Matlab applications – adding an interactive search box to the toolbar of Matlab figures.

The basic idea is simple: whenever the user types in the search box, a Matlab callback function checks the data for the search term. If one or more matches are found then the searchbox’s background remains white, otherwise it is colored yellow to highlight the term. When the user presses <Enter>, the search action is triggered to highlight the term in the data, and any subsequent press of <Enter> will highlight the next match (cycling back at the top as needed). Very simple and intuitive:

Interactive search-box in Matlab figure toolbar

In my specific case, the search action (highlighting the search term in the data) involved doing a lot of work: updating multiple charts and synchronizing row selection in several connected uitables. For this reason, I chose not to do this action interactively (upon each keypress in the search box) but rather only upon clicking <Enter>. In your implementation, if the search action is simpler and faster, you could do it interactively for an even more intuitive effect.

Technical components

The pieces of today’s post were already discussed separately on this website, but never shown together as I will do today:

• The search box component (com.mathworks.widgets.SearchTextField) was discussed in last year’s article on auto-completion widgets
• I showed how to add custom controls to the figure toolbar in a 2009 post (time flies!)
• I discussed controlling toolbar components’ size in another old post
• I discussed my findjobj utility, used for accessing the underlying Java components of Matlab uicontrols in another article
• I discussed Matlab’s use of EDT in a dedicated article on the subject back in 2010 (and I’ll have more to say about this subject next week)
• Finally, I discussed how to trap Java control events in Matlab in separate articles here and here

Adding a search-box to the figure toolbar

As a first step, let’s create the search-box component and add it to our figure’s toolbar:

% First, create the search-box component on the EDT, complete with invokable Matlab callbacks:
jSearch = com.mathworks.widgets.SearchTextField('Symbol');  % 'Symbol' is my default search prompt
jSearchPanel = javaObjectEDT(jSearch.getComponent);  % this is a com.mathworks.mwswing.MJPanel object
jSearchPanel = handle(jSearchPanel, 'CallbackProperties');  % enable Matlab callbacks
 
% Now, set a fixed size for this component so that it does not resize when the figure resizes:
jSize = java.awt.Dimension(100,25);  % 100px wide, 25px tall
jSearchPanel.setMaximumSize(jSize)
jSearchPanel.setMinimumSize(jSize)
jSearchPanel.setPreferredSize(jSize)
jSearchPanel.setSize(jSize)
 
% Now, attach the Matlab callback function to search box events (key-clicks, Enter, and icon clicks):
jSearchBox = handle(javaObjectEDT(jSearchPanel.getComponent(0)), 'CallbackProperties');
set(jSearchBox, 'ActionPerformedCallback', {@searchSymbol,hFig,jSearchBox})
set(jSearchBox, 'KeyPressedCallback',      {@searchSymbol,hFig,jSearchBox})
 
jClearButton = handle(javaObjectEDT(jSearchPanel.getComponent(1)), 'CallbackProperties');
set(jClearButton, 'ActionPerformedCallback', {@searchSymbol,hFig,jSearchBox})
 
% Now, get the handle for the figure's toolbar:
hToolbar = findall(hFig,'tag','FigureToolBar');
jToolbar = get(get(hToolbar,'JavaContainer'),'ComponentPeer');  % or: hToolbar.JavaContainer.getComponentPeer
 
% Now, justify the search-box to the right of the toolbar using an invisible filler control
% (first add the filler control to the toolbar, then the search-box control):
jFiller = javax.swing.Box.createHorizontalGlue;  % this is a javax.swing.Box$Filler object
jToolbar.add(jFiller,      jToolbar.getComponentCount);
jToolbar.add(jSearchPanel, jToolbar.getComponentCount);
 
% Finally, refresh the toolbar so that the new control is displayed:
jToolbar.revalidate
jToolbar.repaint

Search action callback functionality

Now that the control is displayed in the toolbar, let’s define what our Matlab callback function searchSymbol() does. Remember that this callback function is invoked whenever any of the possible events occur: keypress, <Enter>, or clicking the search-box’s icon (typically the “x” icon, to clear the search term).

We first reset the search-box appearance (foreground/background colors), then we check the search term (if non-empty). Based on the selected tab, we search the corresponding data table’s symbol column(s) for the search term. If no match is found, we highlight the search term by setting the search-box’s text to be red over yellow. Otherwise, we change the table’s selected row to the next match’s row index (i.e., the row following the table’s currently-selected row, cycling back at the top of the table if no match is found lower in the table).

Reading and updating the table’s selected row requires using my findjobj utility – for performance considerations the jTable handle should be cached (perhaps in the hTable’s UserData or ApplicationData):

% Callback function to search for a symbol
function searchSymbol(hObject, eventData, hFig, jSearchBox)
    try
        % Clear search-box formatting
        jSearchBox.setBackground(java.awt.Color.white)
        jSearchBox.setForeground(java.awt.Color.black)
        jSearchBox.setSelectedTextColor(java.awt.Color.black)
        jSearchBox.repaint
 
        % Search for the specified symbol in the data table
        symbol = char(jSearchBox.getText);
        if ~isempty(symbol)
            handles = guidata(hFig);
            hTab = handles.hTabGroup.SelectedTab;
            colOffset = 0;
            forceCol0 = false;
            switch hTab.Title
                case 'Scanning'
                    hTable = handles.tbScanResults;
                    symbols = cell(hTable.Data(:,1));
                case 'Correlation'
                    hTable = handles.tbCorrResults;
                    symbols = cell(hTable.Data(:,1:2));
                case 'Backtesting'
                    hTab = handles.hBacktestTabGroup.SelectedTab;
                    hTable = findobj(hTab, 'Type','uitable', 'Tag','results');
                    pairs = cell(hTable.Data(:,1));
                    symbols = cellfun(@(c)strsplit(c,'/'), pairs, 'uniform',false);
                    symbols = reshape([symbols{:}],2,[])';
                    forceCol0 = true;
                case 'Trading'
                    hTable = handles.tbTrading;
                    symbols = cell(hTable.Data(:,2:3));
                    colOffset = 1;
                otherwise  % ignore
                    return
            end
            if isempty(symbols)
                return
            end
            [rows,cols] = ind2sub(size(symbols), find(strcmpi(symbol,symbols)));
            if isempty(rows)
                % Not found - highlight the search term
                jSearchBox.setBackground(java.awt.Color.yellow)
                jSearchBox.setForeground(java.awt.Color.red)
                jSearchBox.setSelectedTextColor(java.awt.Color.red)
                jSearchBox.repaint
            elseif isa(eventData, 'java.awt.event.KeyEvent') && isequal(eventData.getKeyCode,10)
                % Found with <Enter> event - highlight the relevant data row
                jTable = findjobj(hTable);
                try jTable = jTable.getViewport.getView; catch, end  % in case findjobj returns the containing scrollpane rather than the jTable
                [rows, sortedIdx] = sort(rows);
                cols = cols(sortedIdx);
                currentRow = jTable.getSelectedRow + 1;
                idx = find(rows>currentRow,1);
                if isempty(idx),  idx = 1;  end
                if forceCol0
                    jTable.changeSelection(rows(idx)-1, 0, false, false)
                else
                    jTable.changeSelection(rows(idx)-1, cols(idx)-1+colOffset, false, false)
                end
                jTable.repaint
                jTable.getTableHeader.repaint
                jTable.getParent.getParent.repaint
                drawnow
            end
        end
    catch
        % never mind - ignore
    end
end

That’s all there is to it. In my specific case, changing the table’s selected row cased an immediate trigger that updated the associated charts, synchronized the other data tables and did several other background tasks.

What about the new web-based uifigure?

The discussion above refers only to traditional Matlab figures (both HG1 and HG2), not to the new web-based (AppDesigner) uifigures that were officially introduced in R2016a (I wrote about it last year).

AppDesigner uifigures are basically webpages rather than desktop windows (JFrames). They use an entirely different UI mechanism, based on HTML webpages served from a localhost webserver, using the DOJO Javascript toolkit for visualization and interaction, rather than Java Swing as in the existing JFrame figures. The existing figures still work without change, and are expected to continue working alongside the new uifigures for the foreseeable future. I’ll discuss the new uifigures in separate future posts (in the meantime you can read a bit about them in my post from last year).

I suspect that the new uifigures will replace the old figures at some point in the future, to enable a fully web-based (online) Matlab. Will this happen in 2017 or 2027 ? – your guess is as good as mine, but my personal guesstimate is around 2018-2020.

A friend recently asked me, in light of my guesstimate that Java-based Matlab figures will be replaced by web-based figures sometime around 2018-2020, whether there are any “killer features” that make it worthwhile to use undocumented Java-based tricks today, despite the fact that they will probably break in 2-5 years. In my opinion, there are many such features; today I will focus on just a subset of them – those features that relate to the entire figure window.

Over the years I wrote many articles here about figure-level customizations, as well as an entire chapter in my Matlab-Java programming book. So today’s post will be a high-level overview, and users who are interested in any specific topic can visit the referenced links for the implementation details.

An undecorated Matlab figure window – one of many possible figure-level customizations

JavaFrame

JavaFrame is an undocumented hidden property of the figure handle that provides access to the underlying Java window (JFrame) peer object’s reference. Since R2008a, a warning is issued whenever we retrieve this property:

>> jFrame = get(gcf,'JavaFrame');
Warning: figure JavaFrame property will be obsoleted in a future release.
For more information see the JavaFrame resource on the MathWorks web site.
(Type "warning off MATLAB:HandleGraphics:ObsoletedProperty:JavaFrame" to suppress this warning.) 

Until HG2 (R2014b+) we could suppress the warning by simply wrapping the figure handle within a handle() call, as explained here. Since R2014b we need to use the warning function to do this:

warning('off', 'MATLAB:HandleGraphics:ObsoletedProperty:JavaFrame');

We can do several things directly with the JavaFrame‘s properties and methods, including:

• Maximize/minimize/restore the window, via the properties Maximized/Minimized (which accept and return a boolean (logical) value), or the corresponding methods jFrame.isMaximized(), isMinimized(), setMaximized(flag), setMinimized(flag). details
• Modify the container to which the figure will be docked. By default this is the “Figures” container, but this can be changed to any user-specified container, or even to the “Editor”, using the GroupName property or its associated methods. See the related setFigDockGroup utility that I posted on the Matlab File exchange.
• Remove the top separator line between the toolbar and the content-pane, to blend them together, via the jFrame.showTopSeparator(flag) method.
• Retrieve a direct Java reference to the Matlab Desktop and the figure’s internal containers via the Desktop and FigurePanelContainer properties, respectively (we can also get those references by other means).
• Retrieve a direct Java reference to the containing JFrame (Java window), as discussed below
• A few other features that I will not discuss here

MathWorks have set up a dedicated webpage where you can specify how you are using JavaFrame and why it is important for you: http://www.mathworks.com/javaframe. I encourage you to use this webpage to tell MathWorks which features are important for you. This will help them to decide which functionality should be added to the new web-based figures.

JFrame window

The JavaFrame handle enables direct retrieval of the containing Java JFrame (window) reference, using several alternatives. Here are two of these alternatives (there are others):

% Alternative #1
>> jWindow = jFrame.getFigurePanelContainer.getTopLevelAncestor
jWindow = 
com.mathworks.hg.peer.FigureFrameProxy$FigureFrame[fClientProxyFrame,72,62,576x507,...]
 
% Alternative #2
try
    jClient = jFrame.fFigureClient;  % This works up to R2011a
catch
    try
        jClient = jFrame.fHG1Client;  % This works from R2008b-R2014a
    catch
        jClient = jFrame.fHG2Client;  % This works from R2014b and up
    end
end
jWindow = jClient.getWindow;

Customized menu items (top) and figure status bar (bottom)

• Enable/disable the entire figure in a single go (details)
• Remove/restore the window frame (borders and title bar), otherwise known as an “undecorated window” (details)
• Set the figure window to be “Always-On-Top”, i.e. not occluded by any other window, via the AlwaysOnTop property, or the corresponding jWindow.isAlwaysOnTop(), setAlwaysOnTop(flag) methods.
• Make the figure window fully or partially transparent (details). Note: this fails on R2013b/Java7 and higher due to a change in the way that transparency works in Java 7 compared to earlier releases; in other words blame Oracle’s Java, not MathWorks’ Matlab….
• Blur/restore the figure window (details). This too works only up to R2013a.
• Detect and handle window-level focus gain/loss events (details), as well as window-level mouse events (enter/exit/hover etc. – details).
• Customize the figure’s menu bar – dynamic behavior, tooltips, highlights, keyboard shortcuts/accelerators, font colors/styles, callbacks, icons etc. (details1, details2)
• Control figure docking in compiled (deployed) applications (details1, details2)
• Display an integral figure status-bar with text and GUI controls (details1, details2).
• A few other features that I will not discuss here

As you can see, there are numerous very interesting customizations that can be done to Matlab figures which rely on the undocumented implementation. Here are a couple of usage examples that you can easily adapt (follow the links above for additional details and usage examples):

jWindow.setEnabled(false);     % disable entire figure [true/false]
jWindow.setMinimized(true);    % minimize window [true/false]
jWindow.setMaximized(true);    % maximize window [true/false]
jWindow.setAlwaysOnTop(true);  % set to be always on top [true/false]
 
% Set a Matlab callback function to a window focus-gain event
hjWindow = handle(jWindow, 'CallbackProperties');
hjWindow.FocusGainedCallback = @myCallbackFunc;

In addition to the Java-based features above, some functionalities can also be achieved via direct OS manipulations, for example using Jan Simon’s great WindowAPI utility (Windows-only), although I typically prefer using the Java approach since it is cross-platform compatible.

Using all these features is super-easy, so there is not really a question of code complexity or technical risk – the main question is whether to accept the risk that the associated code will stop working when Matlab figures will eventually become web-based.

So is it worth the risk?

This is an excellent question. I contend that the answer depends on the specific use-case. In one project you may decide that it is indeed worth-while to use these undocumented features today, whereas in another GUI you may decide that it is not.

It might make sense to use the features above in any of the following circumstances:

• If you need any of the features in your Matlab GUI today. In this case, you really have no alternative other than to use these features, since there is no documented way to achieve the required functionality.
• If you do not plan to upgrade your Matlab release soon, or at least after the Java-based figures are discontinued in a few years. The commercial Matlab license is perpetual, enabling users to enjoy these features for as long as they continue using this Matlab release.
• If you are compiling your Matlab program using the Matlab Compiler or Coder toolboxes. In such cases, the executable will remain static, until such time (if ever) that you decide to recompile it using a newer Matlab release. Users of the compiled code could continue to use the compiled undocumented features well into the future, for as long as their computers keep running. In such cases, we are not concerned with release compatibility issues.
• If you accept the risk that some recoding may be necessary in the future, or that some functionality will degrade, for the added benefit that they provide your GUIs today.
• If you are willing to code without MathWorks’ official support and endorsement, and accept the fact that they will not fix any internal bugs that you may discover which is related to these features.
• If you wish to present a professional-grade GUI today, and worry about potential incompatibilities only if and when they eventually arrive, sometime in the future.

Here’s another twist to consider: do not take it for granted that when web-based uifigures replace Java-based figures all the documented functionality will work as-is on the new uifigures just as they have on the old figures. In fact, I personally believe that we will need to extensively modify our GUI code to make it compatible with the new uifigures. In other words, avoiding the undocumented hacks above will probably not save us from the need to recode (or at least adapt) our GUI, it will just reduce the necessary work somewhat. We encountered a similar situation with the graphics hacks that I exposed over the years: many people avoided them in the fear that they might someday break; then when R2014b came and HG2 graphics replaced HG1, it turned out that many of these supposedly risky hacks continued working in HG2 (examples: LooseInset, YLimInclude) whereas quite a bit of standard fully-documented Matlab functionality was broken and required some recoding. I believe that the lessons from the HG2 migration were well studied and assimilated by MathWorks, but realistically speaking we should not expect a 100% full-proof transition to uifigures.

Still, accepting the risk does not mean that we should bury our head in the sand. Whenever using any undocumented feature in your code, I strongly suggest to use defensive coding practices, such as wrapping your code within try-catch blocks. This way, even if the feature is removed in R2020a (or whenever), the program will still run, albeit with somewhat diminished functionality, or in other words, graceful degradation. For example:

try
    jFrame = get(hFig, 'JavaFrame');
    jFrame.setMaximized(true);
catch
    oldUnits = get(hFig, 'Units');
    set(hFig, 'Units','norm', 'Pos',[0,0,1,1]);
    set(hFig, 'Units',oldUnits);
end

Once again, I urge you to visit http://www.mathworks.com/javaframe and tell MathWorks which of the above features are important for you. The more users tell MathWorks that they depend on a specific feature, the more would MathWorks be likely to invest R&D efforts in enabling it in the future web-based figures.

Last month, I posted an article that summarized a variety of undocumented customizations to Matlab figure windows. As I noted in that post, Matlab figures have used Java JFrames as their underlying technology since R14 (over a decade ago), but this is expected to change a few years from now with the advent of web-based uifigures. uifigures first became available in late 2014 with the new App Designer preview (the much-awaited GUIDE replacement), and were officially released in R2016a. AppDesigner is actively being developed and we should expect to see exciting new features in upcoming Matlab releases.

Matlab's new AppDesigner (a somewhat outdated screenshot)

However, while AppDesigner has become officially supported, the underlying technology used for the new uifigures remained undocumented. This is not surprising: MathWorks did a good job of retaining backward compatibility with the existing figure handle, and so a new uifigure returns a handle that programmatically appears similar to figure handles, reducing the migration cost when MathWorks decides (presumably around 2018-2020) that web-based (rather than Java-based) figures should become the default figure type. By keeping the underlying figure technology undocumented and retaining the documented top-level behavior (properties and methods of the figure handle), Matlab users who only use the documented interface should expect a relatively smooth transition at that time.

So does this mean that users who start using AppDesigner today (and especially in a few years when web figures become the default) can no longer enjoy the benefits of figure-based customization offered to the existing Java-based figure users (which I listed in last month’s post)? Absolutely not! All we need is to get a hook into the uifigure‘s underlying object and then we can start having fun.

The uifigure Controller

One way to do this is to use the uifigure handle’s hidden (private) Controller property (a matlab.ui.internal.controller.FigureController MCOS object whose source-code appears in %matlabroot%/toolbox/matlab/uitools/uicomponents/components/+matlab/+ui/+internal/+controller/).

Controller is not only a hidden but also a private property of the figure handle, so we cannot simply use the get function to get its value. This doesn’t stop us of course: We can get the controller object using either my getundoc utility or the builtin struct function (which returns private/protected properties as an undocumented feature):

>> hFig = uifigure('Name','Yair', ...);
 
>> figProps = struct(hFig);  % or getundoc(hFig)
Warning: Calling STRUCT on an object prevents the object from hiding its implementation details and should thus be
avoided. Use DISP or DISPLAY to see the visible public details of an object. See 'help struct' for more information.
(Type "warning off MATLAB:structOnObject" to suppress this warning.)
 
Warning: figure JavaFrame property will be obsoleted in a future release. For more information see
the JavaFrame resource on the MathWorks web site.
(Type "warning off MATLAB:HandleGraphics:ObsoletedProperty:JavaFrame" to suppress this warning.)
 
figProps = 
                      JavaFrame: []
                    JavaFrame_I: []
                       Position: [87 40 584 465]
                   PositionMode: 'auto'
                            ...
                     Controller: [1x1 matlab.ui.internal.controller.FigureController]
                 ControllerMode: 'auto'
                            ...
 
>> figProps.Controller
ans = 
  FigureController with properties:
 
       Canvas: []
    ProxyView: [1x1 struct]
 
>> figProps.Controller.ProxyView
ans = 
            PeerNode: [1x1 com.mathworks.peermodel.impl.PeerNodeImpl]
    PeerModelManager: [1x1 com.mathworks.peermodel.impl.PeerModelManagerImpl]
 
>> struct(figProps.Controller)
Warning: Calling STRUCT on an object prevents the object from hiding its implementation details and should thus be
avoided. Use DISP or DISPLAY to see the visible public details of an object. See 'help struct' for more information.
(Type "warning off MATLAB:structOnObject" to suppress this warning.)
 
ans = 
               PositionListener: [1x1 event.listener]
    ContainerPositionCorrection: [1 1 0 0]
                      Container: [1x1 matlab.ui.internal.controller.FigureContainer]
                         Canvas: []
                  IsClientReady: 1
              PeerEventListener: [1x1 handle.listener]
                      ProxyView: [1x1 struct]
                          Model: [1x1 Figure]
               ParentController: [0x0 handle]
      PropertyManagementService: [1x1 matlab.ui.internal.componentframework.services.core.propertymanagement.PropertyManagementService]
          IdentificationService: [1x1 matlab.ui.internal.componentframework.services.core.identification.WebIdentificationService]
           EventHandlingService: [1x1 matlab.ui.internal.componentframework.services.core.eventhandling.WebEventHandlingService]

I will discuss all the goodies here in a future post (if you are curious then feel free to start drilling in there yourself, I promise it won’t bite you…). However, today I wish to concentrate on more immediate benefits from a different venue:

The uifigure webwindow

uifigures are basically webpages rather than desktop windows (JFrames). They use an entirely different UI mechanism, based on HTML webpages served from a localhost webserver that runs CEF (Chromium Embedded Framework version 3.2272 on Chromium 41 in R2016a). This runs the so-called CEF client (apparently an adaptation of the CefClient sample application that comes with CEF; the relevant Matlab source-code is in %matlabroot%/toolbox/matlab/cefclient/). It uses the DOJO Javascript toolkit for UI controls visualization and interaction, rather than Java Swing as in the existing JFrame figures. I still don’t know if there is a way to combine the seemingly disparate sets of GUIs (namely adding Java-based controls to web-based figures or vice-versa).

Anyway, the important thing to note for my purposes today is that when a new uifigure is created, the above-mentioned Controller object is created, which in turn creates a new matlab.internal.webwindow. The webwindow class (%matlabroot%/toolbox/matlab/cefclient/+matlab/+internal/webwindow.m) is well-documented and easy to follow (although the non camel-cased class name escaped someone’s attention), and allows access to several important figure-level customizations.

The figure’s webwindow reference can be accessed via the Controller‘s Container‘s CEF property:

>> hFig = uifigure('Name','Yair', ...);
>> warning off MATLAB:structOnObject      % suppress warning (yes, we know it's naughty...)
>> figProps = struct(hFig);
 
>> controller = figProps.Controller;      % Controller is a private hidden property of Figure
>> controllerProps = struct(controller);
 
>> container = controllerProps.Container  % Container is a private hidden property of FigureController
container = 
  FigureContainer with properties:
 
    FigurePeerNode: [1x1 com.mathworks.peermodel.impl.PeerNodeImpl]
         Resizable: 1
          Position: [86 39 584 465]
               Tag: ''
             Title: 'Yair'
              Icon: 'C:\Program Files\Matlab\R2016a\toolbox\matlab\uitools\uicomponents\resources\images…'
           Visible: 1
               URL: 'http://localhost:31417/toolbox/matlab/uitools/uifigureappjs/componentContainer.html…'
              HTML: 'toolbox/matlab/uitools/uifigureappjs/componentContainer.html'
     ConnectorPort: 31417
         DebugPort: 0
     IsWindowValid: 1
 
>> win = container.CEF   % CEF is a regular (public) hidden property of FigureContainer
win = 
  webwindow with properties:
 
                             URL: 'http://localhost:31417/toolbox/matlab/uitools/uifigureappjs/component…'
                           Title: 'Yair'
                            Icon: 'C:\Program Files\Matlab\R2016a\toolbox\matlab\uitools\uicomponents\re…'
                        Position: [86 39 584 465]
     CustomWindowClosingCallback: @(o,e)this.Model.hgclose()
    CustomWindowResizingCallback: @(event,data)resizeRequest(this,event,data)
                  WindowResizing: []
                   WindowResized: []
                     FocusGained: []
                       FocusLost: []
                DownloadCallback: []
        PageLoadFinishedCallback: []
           MATLABClosingCallback: []
      MATLABWindowExitedCallback: []
             PopUpWindowCallback: []
             RemoteDebuggingPort: 0
                      CEFVersion: '3.2272.2072'
                 ChromiumVersion: '41.0.2272.76'
                   isWindowValid: 1
               isDownloadingFile: 0
                         isModal: 0
                  isWindowActive: 1
                   isAlwaysOnTop: 0
                     isAllActive: 1
                     isResizable: 1
                         MaxSize: []
                         MinSize: []
 
>> win.URL
ans =
http://localhost:31417/toolbox/matlab/uitools/uifigureappjs/componentContainer.html?channel=/uicontainer/393ed66a-5e34-41f3-8ac0-0b0f3b0738cd&snc=5C2353

An alternative way to get the webwindow is via the list of all webwindows stored by a central webwindowmanager:

webWindows = matlab.internal.webwindowmanager.instance.findAllWebwindows();  % manager method returning an array of all open webwindows
webWindows = matlab.internal.webwindowmanager.instance.windowList;           % equivalent alternative via manager's windowList property

Note that the controller, container and webwindow class objects, like most Matlab MCOS objects, have internal (hidden) properties/methods that you can explore. For example:

>> getundoc(win)
ans = 
                   Channel: [1x1 asyncio.Channel]
       CustomEventListener: [1x1 event.listener]
           InitialPosition: [100 100 600 400]
    JavaScriptReturnStatus: []
     JavaScriptReturnValue: []
     NewWindowBeingCreated: 0
          NewWindowCreated: 1
           UpdatedPosition: [86 39 584 465]
              WindowHandle: 2559756
                    newURL: 'http://localhost:31417/toolbox/matlab/uitools/uifigureappjs/componentContai…'

Using webwindow for figure-level customizations

We can use the methods of this webwindow object as follows:

win.setAlwaysOnTop(true);   % always on top of other figure windows (a.k.a. AOT)
 
win.hide();
win.show();
win.bringToFront();
 
win.minimize();
win.maximize();
win.restore();
 
win.setMaxSize([400,600]);  % enables resizing up to this size but not larger (default=[])
win.setMinSize([200,300]);  % enables resizing down to this size but not smaller (default=[])
win.setResizable(false);
 
win.setWindowAsModal(true);
 
win.setActivateCurrentWindow(false);  % disable interaction with this entire window
win.setActivateAllWindows(false);     % disable interaction with *ALL* uifigure (but not Java-based) windows
 
result = win.executeJS(jsStr, timeout);  % run JavaScript

In addition to these methods, we can set callback functions to various callbacks exposed by the webwindow as regular properties (too bad that some of their names [like the class name itself] don’t follow Matlab’s standard naming convention, in this case by appending “Fcn” or “Callback”):

win.FocusGained = @someCallbackFunc;
win.FocusLost = @anotherCallbackFunc;

In summary, while the possible customizations to Java-based figure windows are more extensive, the webwindow methods appear to cover most of the important ones. Since these functionalities (maximize/minimize, AOT, disable etc.) are now common to both the Java and web-based figures, I really hope that MathWorks will create fully-documented figure properties/methods for them. Now that there is no longer any question whether these features will be supported by the future technology, and since there is no question as to their usefulness, there is really no reason not to officially support them in both figure types. If you feel the same as I do, please let MathWorks know about this – if enough people request this, MathWorks will be more likely to add these features to one of the upcoming Matlab releases.

Warning: the internal implementation is subject to change across releases, so be careful to make your code cross-release compatible whenever you rely on one of Matlab’s internal objects.

Note that I labeled this post as “part 1” – I expect to post additional articles on uifigure customizations in upcoming years.

I would like to introduce guest blogger Iliya Romm of Israel’s Technion Turbomachinery and Heat Transfer Laboratory. Today Iliya will discuss how Matlab’s new web-based figures can be customized with user-controlled CSS and JavaScript code.

When we compare the documented properties of a “classic” uicontrol with an App Designer control such as uicheckbox, we see lists of 42 and 15 properties, respectively. At first glance, this implies that our ability to customize App Designer elements is relatively very limited. This is surely a disquieting conclusion, especially for those used to being able to change most aspect of their Matlab figures via Java. Fortunately, such a conclusion is quite far from reality, as we will shortly see.

To understand this claim, we need to consider a previous post on this blog, where Yair discussed how uifigures are actually HTML webpages rendered by Matlab. As such, they have a DOM that can be accessed and manipulated through JavaScript commands to achieve various visual customizations. Today we’ll explore the structure of the uifigure webpage; take a look at some possibilities provided by the Dojo Toolkit; and see how to use Dojo to customize uifigure controls visually using CSS styles and/or HTML attributes.

User customizations of Matlab uifigures (click to zoom-in)

A brief introduction to CSS

CSS stands for Cascading Style Sheets. As described on the official webpage of W3C (which governs web standards):

CSS is the language for describing the presentation of Web pages, including colors, layout, and fonts. CSS is independent of HTML. This is referred to as the separation of structure (or: content) from presentation.

CSS rules (or “styles”) can be defined in one of three places:

• A separate file, such as the main.css that Matlab uses for uifigures (this file is found minified in %matlabroot%\toolbox\matlab\uitools\uifigureappjs\release\gbtclient\css)
• An inline block inside the HTML’s <head> section
• Directly within a DOM node

Deciding which of the above to use, is largely a choice of the right tool for the job. Usually, the first two choices should be preferred, as they adhere to the “separation of structure and presentation” idea better. However, in the scope of this demonstration, we’ll be using mostly the 3rd option, because it allows us not to worry about possible CSS precedence issues (suggested read).

The syntax of CSS is generally: selector { property: value }, but it can have other forms as well.

Getting down to business

Let us consider a very basic uifigure that only contains a uitextarea and its label:

Simple demo uifigure with a TextArea and label

The auto-generated code for it is:

classdef DOMdemo < matlab.apps.AppBase
 
    % Properties that correspond to app components
    properties (Access = public)
        UIFigure      matlab.ui.Figure           % UI Figure
        LabelTextArea matlab.ui.control.Label    % Text Area
        TextArea      matlab.ui.control.TextArea % This is some text.        
    end
 
    methods (Access = private)
        % Code that executes after component creation
        function startupFcn(app)
        end
    end
 
    % App initialization and construction
    methods (Access = private)
 
        % Create UIFigure and components
        function createComponents(app)
            % Create UIFigure
            app.UIFigure = uifigure;
            app.UIFigure.Position = [100 100 280 102];
            app.UIFigure.Name = 'UI Figure';
            setAutoResize(app, app.UIFigure, true)
 
            % Create LabelTextArea
            app.LabelTextArea = uilabel(app.UIFigure);
            app.LabelTextArea.HorizontalAlignment = 'right';
            app.LabelTextArea.Position = [16 73 62 15];
            app.LabelTextArea.Text = 'Text Area';
 
            % Create TextArea
            app.TextArea = uitextarea(app.UIFigure);
            app.TextArea.Position = [116 14 151 60];
            app.TextArea.Value = {'This is some text.'};
        end
    end
 
    methods (Access = public)
 
        % Construct app
        function app = DOMdemo()
            % Create and configure components
            createComponents(app)
 
            % Register the app with App Designer
            registerApp(app, app.UIFigure)
 
            % Execute the startup function
            runStartupFcn(app, @startupFcn)
 
            if nargout == 0
                clear app
            end
        end
 
        % Code that executes before app deletion
        function delete(app)
            % Delete UIFigure when app is deleted
            delete(app.UIFigure)
        end
    end
end

Let’s say we want to modify certain aspects of the TextArea widget, such as the text color, background, and/or horizontal alignment. The workflow for styling elements involves:

1. Find the handle to the webfigure
2. Find the DOM node we want to modify
3. Find the property name that corresponds to the change we want
4. Find a way to manipulate the desired node from Matlab

Step 1: Find the handle to the webfigure

The first thing we need to do is to strategically place a bit of code that would allow us to get the URL of the figure so we can inspect it in our browser:

function startupFcn(app)
   % Customizations (aka "MAGIC GOES HERE"):
   warning off Matlab:HandleGraphics:ObsoletedProperty:JavaFrame
   warning off Matlab:structOnObject    
   while true
      try   
         win = struct(struct(struct(app).UIFigure).Controller).Container.CEF;
         disp(win.URL);
         break
      catch
         disp('Not ready yet!');
         pause(0.5); % Give the figure (webpage) some more time to load
      end
   end
end

This code waits until the page is sufficiently loaded, and then retrieve its local address (URL). The result will be something like this, which can be directly opened in any browser (outside Matlab):

http://localhost:31415/toolbox/matlab/uitools/uifigureappjs/componentContainer.html?channel=/uicontainer/861ef484-534e-4a50-993e-6d00bdba73a5&snc=88E96E

Step 2: Find the DOM node that corresponds to the component that we want to modify

Loading this URL in an external browser (e.g., Chrome, Firefox or IE/Edge) enables us to use web-development addins (e.g., FireBug) to inspect the page contents (source-code). Opening the URL inside a browser and inspecting the page contents, we can see its DOM:

Inspecting the DOM in Firefox (click to zoom-in)

Notice the three data-tag entries marked by red frames. Any idea why there are exactly three nonempty tags like that? This is because our App Designer object, app, contains 3 declared children, as defined in:

createComponents(app):
    app.UIFigure = uifigure;
    app.LabelTextArea = uilabel(app.UIFigure);
    app.TextArea = uitextarea(app.UIFigure);

… and each of them is assigned a random hexadecimal id whenever the app is opened.

Finding the relevant node involved some trial-and-error, but after doing it several times I seem to have found a consistent pattern that can be used to our advantage. Apparently, the nodes with data-tag are always above the element we want to style, sometimes as a direct parent and sometimes farther away. So why do we even need to bother with choosing more accurate nodes than these “tagged” ones? Shouldn’t styles applied to the tagged nodes cascade down to the element we care about? Sure, sometimes it works like that, but we want to do better than “sometimes”. To that end, we would like to select as relevant a node as possible.

Anyway, the next step in the program is to find the data-tag that corresponds to the selected component. Luckily, there is a direct (undocumented) way to get it:

% Determine the data-tag of the DOM component that we want to modify:
hComponent = app.TextArea;  % handle to the component that we want to modify
data_tag = char(struct(hComponent).Controller.ProxyView.PeerNode.getId);  % this part is generic: can be used with any web-based GUI component

Let’s take a look at the elements marked with blue and green borders (in that order) in the DOM screenshot. We see that the data-tag property is exactly one level above these elements, in other words, the first child of the tagged node is an element that contains a widgetid property. This property is very important, as it contains the id of the node that we actually want to change. Think pointers. To summarize this part:

We shall use this transformation in Step 4 below.

I wanted to start with the blue-outlined element as it demonstrates this structure using distinct elements. The green-outlined element is slightly strange, as it contains a widgetid that points back to itself. Since this obeys the same algorithm, it’s not a problem.

Step 3: Find the CSS property name that corresponds to the change we want

There is no trick here: it’s just a matter of going through a list of CSS properties and choosing one that “sounds about right” (there are often several ways to achieve the same visual result with CSS). After we choose the relevant properties, we need to convert them to camelCase as per documentation of dojo.style():

If the CSS style property is hyphenated, the JavaScript property is camelCased. For example: “font-size” becomes “fontSize”, and so on.

Note that Matlab R2016a comes bundled with Dojo v1.10.4, rev. f4fef70 (January 11 2015). Other Matlab releases will probably come with other Dojo versions. They will never be the latest version of Dojo, but rather a version that is 1-2 years old. We should keep this in mind when searching the Dojo documentation. We can get the current Dojo version as follows:

>> f=uifigure; drawnow; dojoVersion = matlab.internal.webwindowmanager.instance.windowList(1).executeJS('dojo.version'), delete(f)
dojoVersion =
{"major":1,"minor":10,"patch":4,"flag":"","revision":"f4fef70"}

This tells us that Dojo 1.10.4.f4fef70 is the currently-used version. We can use this information to browse the relevant documentation branch, as well as possibly use different Dojo functions/features.

Step 4: Manipulate the desired element from Matlab

In this demo, we’ll use a combination of several commands:

• {matlab.internal.webwindow.}executeJS() – For sending JS commands to the uifigure.
• dojo.query() – for finding nodes inside the DOM.
• dojo.style() (deprecated since v1.8) – for applying styles to the required nodes of the DOM.
  Syntax: dojo.style(node, style, value);
• dojo.setAttr (deprecated since v1.8) – for setting some non-style attributes.
  Syntax: dojo.setAttr(node, name, value);

Consider the following JS commands:

• search the DOM for nodes having a data-tag attribute having the specified value, take their first child of type <div>, and return the value of this child’s widgetid attribute:

  ['dojo.getAttr(dojo.query("[data-tag^=''' data_tag '''] > div")[0],"widgetid")']

• search the DOM for nodes with id of widgetid, then take the first element of the result and set its text alignment:

  ['dojo.style(dojo.query("#' widgetId(2:end-1) '")[0],"textAlign","center")']

• append the CSS style defined by {SOME CSS STYLE} to the page (this style can later be used by nodes):

  ['document.head.innerHTML += ''<style>{SOME CSS STYLE}</style>''']);

Putting it all together

It should finally be possible to understand the code that appears in the animated screenshot at the top of this post:

%% 1. Get a handle to the webwindow:
win = struct(struct(struct(app).UIFigure).Controller).Container.CEF;
 
%% 2. Find which element of the DOM we want to edit (as before):
data_tag = char(struct(app.TextArea).Controller.ProxyView.PeerNode.getId);
 
%% 3. Manipulate the DOM via a JS command
% ^ always references a class="vc-widget" element.
widgetId = win.executeJS(['dojo.getAttr(dojo.query("[data-tag^=''' data_tag '''] > div")[0],"widgetid")']);
 
% Change font weight:
dojo_style_prefix = ['dojo.style(dojo.query("#' widgetId(2:end-1) '")[0],'];
win.executeJS([dojo_style_prefix '"fontWeight","900")']);
 
% Change font color:
win.executeJS([dojo_style_prefix '"color","yellow")']);
 
% Add an inline css to the HTML <head>:
win.executeJS(['document.head.innerHTML += ''<style>'...
    '@-webkit-keyframes mymove {50% {background-color: blue;}}'...
    '@keyframes mymove {50% {background-color: blue;}}</style>''']);
 
% Add animation to control:      
win.executeJS([dojo_style_prefix '"-webkit-animation","mymove 5s infinite")']);
 
% Change Dojo theme:
win.executeJS('dojo.setAttr(document.body,''class'',''nihilo'')[0]');
 
% Center text:
win.executeJS([dojo_style_prefix '"textAlign","center")']);

A similar method for center-aligning the items in a uilistbox is described here (using a CSS text-align directive).

The only thing we need to ensure before running code that manipulates the DOM, is that the page is fully loaded. The easiest way is to include a pause() of several seconds right after the createComponents(app) function (this will not interfere with the creation of the uifigure, as it happens on a different thread). I have been experimenting with another method involving webwindow‘s PageLoadFinishedCallback callback, but haven’t found anything elegant yet.

A few words of caution

In this demonstration, we invoked Dojo functions via the webwindow’s JS interface. For something like this to be possible, there has to exist some form of “bridge” that translates Matlab commands to JS commands issued to the browser and control the DOM. We also know that this bridge has to be bi-directional, because binding Matlab callbacks to uifigure actions (e.g. ButtonPushFcn for uibuttons) is a documented feature.

The extent to which the bridge might allow malicious code to control the Matlab process needs to be investigated. Until then, the ability of webwindows to execute arbitrary JS code should be considered a known vulnerability. For more information, see XSS and related vulnerabilities.

Final remarks

It should be clear now that there are actually lots of possibilities afforded by the new uifigures for user customizations. One would hope that future Matlab releases will expose easier and more robust hooks for CSS/JS customizations of uifigure contents. But until that time arrives (if ever), we can make do with the mechanism shown above.

Readers are welcome to visit the GitHub project dedicated to manipulating uifigures using the methods discussed in this post. Feel free to comment, suggest improvements and ideas, and of course submit some pull requests

p.s. – it turns out that uifigures can also display MathML. But this is a topic for another post…

With high-density displays becoming increasingly popular, some users set their display’s DPI to a higher-than-standard (i.e., >100%) value, in order to compensate for the increased pixel density to achieve readable interfaces. This OS setting tells the running applications that there are fewer visible screen pixels, and these are spread over a larger number of physical pixels. This works well for most cases (at least on recent OSes, it was a bit buggy in non-recet ones). Unfortunately, in some cases we might actually want to know the screen size in physical, rather than logical, pixels. Apparently, Matlab root’s ScreenSize property only reports the logical (scaled) pixel size, not the physical (unscaled) one:

>> get(0,'ScreenSize')   % with 100% DPI (unscaled standard)
ans =
        1       1      1366       768
 
>> get(0,'ScreenSize')   % with 125% DPI (scaled)
ans =
        1       1      1092.8     614.4

The same phenomenon also affects other related properties, for example MonitorPositions.

Raimund Schlüßler, a reader on this blog, was kind enough to point me to this problem and its workaround, which I thought worthy to share here: To get the physical screen-size, use the following builtin Java command:

>> jScreenSize = java.awt.Toolkit.getDefaultToolkit.getScreenSize
jScreenSize =
java.awt.Dimension[width=1366,height=768]
 
>> width = jScreenSize.getWidth
width =
        1366
 
>> height = jScreenSize.getHeight
height =
        768

Upcoming travels – London/Belfast, Zürich & Geneva

I will shortly be traveling to consult some clients in Belfast (via London), Zürich and Geneva. If you are in the area and wish to meet me to discuss how I could bring value to your work, then please email me (altmany at gmail):

• Belfast: Nov 28 – Dec 1 (flying via London)
• Zürich: Dec 11-12
• Geneva: Dec 13-15

Over the years I’ve reported numerous uses for integrating Java components and functionality in Matlab. As I’ve also recently reported, MathWorks is apparently making a gradual shift away from standalone Java-based figures, toward browser-based web-enabled figures. As I surmised a few months ago, MathWorks has created dedicated surveys to solicit user feedbacks on the most important (and undocumented) non-compatible aspects of this paradigm change: one regarding users’ use of the javacomponent function, the other regarding the use of the figure’s JavaFrame property:

• Survey regarding usage of the javacomponent function: http://www.mathworks.com/javacomponent
• Survey regarding usage of the JavaFrame property: http://www.mathworks.com/javaframe

In MathWorks’ words:

In order to extend your ability to build MATLAB apps, we understand you sometimes need to make use of undocumented Java UI technologies, such as the JavaFrame property. In response to your needs, we are working to develop documented alternatives that address gaps in our app building offerings.

To help inform our work and plans, we would like to understand how you are using the JavaFrame property. Based on your understanding of how it is being used within your app, please take a moment to fill out the following survey. The survey will take approximately 1-2 minutes to finish.

I urge anyone who uses one or both of these features to let MathWorks know how you’re using them, so that they could incorporate that functionality into the core (documented) Matlab. The surveys are really short and to the point. If you wish to send additional information, please email George.Caia at mathworks.com.

The more feedback responses that MathWorks will get, the better it will be able to prioritize its R&D efforts for the benefit of all users, and the more likely are certain features to get a documented solution at some future release. If you don’t take the time now to tell MathWorks how you use these features in your code, don’t complain if and when they break in the future…

My personal uses of these features

• Functionality:
  • Figure: maximize/minimize/restore, enable/disable, always-on-top, toolbar controls, menu customizations (icons, tooltips, font, shortcuts, colors)
  • Table: sorting, filtering, grouping, column auto-sizing, cell-specific behavior (tooltip, context menu, context-sensitive editor, merging cells)
  • Tree control
  • Listbox: cell-specific behavior (tooltip, context menu)
  • Tri-state checkbox
  • uicontrols in general: various event callbacks (e.g. mouse hover/unhover, focus gained/lost)
  • Ability to add Java controls e.g. color/font/date/file selector panel or dropdown, spinner, slider, search box, password field
  • Ability to add 3rd-party components e.g. JFreeCharts, JIDE controls/panels

• Appearance:
  • Figure: undecorated (frameless), other figure frame aspects
  • Table: column/cell-specific rendering (alignment, icons, font, fg/bg color, string formatting)
  • Listbox: auto-hide vertical scrollbar as needed, cell-specific renderer (icon, font, alignment, fg/bg color)
  • Button/checkbox/radio: icons, text alignment, border customization, Look & Feel
  • Right-aligned checkbox (button to the right of label)
  • Panel: border customization (rounded/matte/…)

You can find descriptions/explanations of many of these in posts I made on this website over the years.

As I have repeatedly posted in recent years, Matlab is advancing towards web-based GUI. The basic underlying technology is more-or-less stable: an HTML/Javascript webpage that is created-on-the-fly and rendered in a stripped-down browser window (based on Chromium-based jxBrowser in recent years). However, the exact mechanism by which the controls (“widgets”) are actually converted into visible components (currently based on the Dojo toolkit and its Dijit UI library) and interact with Matlab (i.e., the internal Matlab class structures that interact with the browser and Dojo) is still undergoing changes and is not quite as stable.

Customization hacks reported on this blog last year (part 1, part 2) may fail in some cases due to the changing nature of the undocumented internals. Some examples are the way by which we can extract the uifigure’s URL (which changed in R2017a), the ability to display and debug uifigures in a standard webbrowser with associated dev tools (which seems to have stopped working in R2017b), and the way by which we can extract the Dijit reference of displayed uicontrols.

Greatly assisting in this respect is Iliya Romm, who was the guest blogger for part 2 of this series last year. Iliya co-authored the open-source (GitHub) mlapptools toolbox, which enables accessing and customizing uifigure components using standard CSS, without users having to bother about the ugly hacks discussed in the previous parts of the series. This toolbox is really just a single Matlab class (mlapptools), contained within a single m-file (mlapptools.m). In addition to this class, the toolbox includes a README.md mark-down usage documentation, and two demo functions, DOMdemoGUI.m and TableDemo.m.

Here is the effect of using TableDemo, that shows how we can customize individual uitable cells (each uitable cell is a separate Dijit widget that can be customized individually):

CSS customizations of uifigure components

The mlapptools class contains several static methods that can be used individually:

• textAlign(uielement, alignment) – Modify text horizontal alignment ('left', 'center', 'right', 'justify' or 'initial')
• fontWeight(uielement, weight) – Modify font weight ('normal', 'bold', 'bolder', 'lighter' or 'initial'), depending on availability in the font-face used
• fontColor(uielement, color) – Modify font color (e.g. 'red', '#ff0000', 'rgb(255,0,0)' or other variants)
• setStyle(uielement, styleAttr, styleValue) – Modify a specified CSS style attribute
• aboutDojo() – Return version information about the Dojo toolkit
• getHTML(hFig) – Return the full HTML code of a uifigure
• getWebWindow(hFig) – Return a webwindow handle from a uifigure handle
• getWebElements (hControl) – Return a webwindow handle and a widget ID for the specified uicontrol handle
• getWidgetList(hFig, verboseFlag) – Return a cell-array of structs containing information about all widgets in the uifigure
• getWidgetInfo(hWebwindow, widgetId, verboseFlag) – Return information about a specific dijit widget
• setTimeout(hFig, seconds) – Override the default timeout (=5 secs) for dojo commands, for a specific uifigure

A few simple usage examples:

mlapptools.fontColor(hButton,'red')  % set red text color
mlapptools.fontWeight(hButton,'bold')  % set bold text font
mlapptools.setStyle(hButton,'border','2px solid blue')  % add a 2-pixel solid blue border
mlapptools.setStyle(hButton,'background-image','url(https://www.mathworks.com/etc/designs/mathworks/img/pic-header-mathworks-logo.svg)')  % add background image

Once you download mlapptools and add its location to the Matlab path, you can use it in any web-based GUI that you create, either programmatically or with Add-Designer.

The mlapptools is quite well written and documented, so if you are interested in the inner workings I urge you to take a look at this class’s private methods. For example, to understand how a Matlab uicontrol handle is converted into a Dojo widget-id, which is then used with the built-in dojo.style() Javascript function to modify the CSS attributes of the HTML <div> or <span> that are the control’s visual representation on the webpage. An explanation of the underlying mechanism can be found in part 2 of this series of articles on uifigure customizations. Note that the mlapptools code is newer than the article and contains some new concepts that were not covered in that article, for example searching through Dijit’s registry of displayed widgets.

Note: web-based GUI is often referred to as “App-Designed” (AD) GUI, because using the Matlab App Designer is the typical way to create and customize such GUIs. However, just as great-looking GUIs could be created programmatically rather than with GUIDE, so too can web-based GUIS be created programmatically, using regular built-in Matlab commands such as uifigure, uibutton and uitable (an example of such programmatic GUI creation can be found in Iliya’s TableDemo.m, discussed above). For this reason, I believe that the new GUIs should be referred to as “uifigures” or “web GUIs”, and not as “AD GUIs”.

If you have any feature requests or bugs related to mlapptools, please report them on its GitHub issues page. For anything else, please add a comment below.

8 years ago I blogged about Matlab’s builtin HelpPopup widget. This control is used by Matlab to display popup-windows with help documentation, but can also be used by users to display custom lightweight popups that contain HTML-capable text and even URLs of entire webpages. Today I’d like to highlight another builtin Matlab widget, ctrluis.PopupPanel, which can be used to display rich contents in a lightweight popup box attached to a specific Matlab figure:

Matlab's builtin PopupPanel widget

As you can see, this popup-panel displays richly-formatted contents, having either an opaque or transparent background, with vertical scrollbars being applied automatically. The popup pane is not limited to displaying text messages – in fact, it can display any Java GUI container (e.g. a settings panel). This popup-panel is similar in concept to the HelpPopup widget, and yet much more powerful in several aspects.

Creating the popup panel

Creating a PopupPanel is very simple:

% Create the popup-panel in the specified figure
hPopupPanel = ctrluis.PopupPanel(gcf);  % use gcf or any figure handle
hPopupPanel.setPosition([.1,.1,.8,.8]);  % set panel position (normalized units)
 
% Alternative #1: set popup-panel's contents to some HTML-formatted message
% note: createMessageTextPane() has optional input args FontName (arg #2), FontSize (#3)
jPanel = ctrluis.PopupPanel.createMessageTextPane('testing <b><i>123</i></b> ...')
hPopupPanel.setPanel(jPanel);
 
% Alternative #2: set popup-panel's contents to a webpage URL
url = 'http://undocumentedmatlab.com/files/sample-webpage.html';
jPanel = javaObjectEDT(javax.swing.JEditorPane(url));
hPopupPanel.setPanel(jPanel);

The entire contents are embedded within a scroll-box (which is a com.mathworks.widgets.LightScrollPane object) whose scrollbars automatically appear as-needed, so we don’t need to worry about the contents fitting the allocated space.

To display custom GUI controls in the popup, we can simply contain those GUI controls in a Java container (e.g., a JPanel) and then do hPopupPanel.setPanel(jPanel). This functionality can be used to create unobtrusive settings panels, input dialogs etc.

The nice thing about the popup widget is that it is attached to the figure, and yet is not assigned a heavyweight window (so it does not appear in the OS task-bar). The popup moves along with the figure when the figure is moved, and is automatically disposed when the figure is closed.

A few caveats about the ctrluis.PopupPanel control:

• The widget’s parent is expected to be a figure that has pixel units. If it doesn’t, the internal computations of ctrluis.PopupPanel croak.
• The widget’s position is specified in normalized units (default: [0,0,1,1]). This normalized position is only used during widget creation: after creation, if you resize the figure the popup-panel’s position remains unchanged. To modify/update the position of the popup-panel programmatically, use hPopupPanel.setPosition(newPosition). Alternatively, update the control’s Position property and then call hPopupPanel.layout() (there is no need to call layout when you use setPosition).
• This functionality is only available for Java-based figures, not the new web-based (AppDesigner) uifigures.

Popup panel customizations

We can open/close the popup panel by clicking on its icon, as shown in the screenshots above, or programmatically using the control’s methods:

% Programmatically open/close the popup-panel
hPopupPanel.showPanel;
hPopupPanel.hidePanel;
 
% Show/hide entire popup-panel widget (including its icon)
hPopupPanel.setVisible(true);   % or .setVisible(1) or .Visible=1
hPopupPanel.setVisible(false);  % or .setVisible(0) or .Visible=0

To set a transparent background to the popup-panel (as shown in the screenshots above), we need to unset the opacity of the displayed panel and several of its direct parents:

% Set a transparent popup-panel background
for idx = 1 : 6
   jPanel.setOpaque(false);  % true=opaque, false=transparent
   jPanel = jPanel.getParent;
end
jPanel.repaint

Note that in the screenshots above, the panel’s background is made transparent, but the contained text and image remain opaque. Your displayed images can of course contain transparency and animation, if this is supported by the image format (for example, GIF).

iptui.internal.utilities.addMessagePane

ctrluis.PopupPanel is used internally by iptui.internal.utilities.addMessagePane(hFig,message) in order to display a minimizable single-line message panel at the top of a specified figure:

hPopupPanel = iptui.internal.utilities.addMessagePane(gcf, 'testing <b>123</b> ...');  % note the HTML formatting

The function updates the message panel’s position whenever the figure’s size is modified (by trapping the figure’s SizeChangedFcn), to ensure that the panel is always attached to the top of the figure and spans the full figure width. This is a simple function so I encourage you to take a look at its code (%matlabroot%/toolbox/images/imuitools/+iptui/+internal/+utilities/addMessagePane.m) – note that this might require the Image Processing Toolbox (I’m not sure).

Matlab's builtin iptui.internal.utilities.addMessagePane

Professional assistance anyone?

As shown by this and many other posts on this site, a polished interface and functionality is often composed of small professional touches, many of which are not exposed in the official Matlab documentation for various reasons. So if you need top-quality professional appearance/functionality in your Matlab program, or maybe just a Matlab program that is dependable, robust and highly-performant, consider employing my consulting services.

Last week, a Matlab user asked whether it is possible to customize the context (right-click) menu that is presented in plot-edit mode. This menu is displayed by clicking the plot-edit (arrow) icon on the standard Matlab figure toolbar, then right-clicking any graphic/GUI element in the figure. Unfortunately, it seems that this context menu is only created the first time that a user right-clicks in plot-edit mode – it is not accessible before then, and so it seems impossible to customize the menu before it is presented to the user the first time.

Customized plot-edit context-menu

A few workarounds were suggested to the original poster and you are most welcome to review them. There is also some discussion about the technical reasons that none of the “standard” ways of finding and modifying menu items fail in this case.

In today’s post I wish to repost my solution, in the hope that it might help other users in similar cases.

My solution is basically this:

1. First, enter plot-edit mode programmatically using the plotedit function
2. Next, move the mouse to the screen location of the relevant figure component (e.g. axes). This can be done in several different ways (the root object’s PointerLocation property, the moveptr function, or java.awt.Robot.mouseMove() method).
3. Next, automate a mouse right-click using the built in java.awt.Robot class (as discussed in this blog back in 2010)
4. Next, locate the relevant context-menu item and modify its label, callback or any of its other properties
5. Next, dismiss the context-menu by simulating a follow-on right-click using the same Robot object
6. Finally, exit plot-edit mode and return the mouse pointer to its original location

% Create an initial figure / axes for demostration purpose
fig = figure('MenuBar','none','Toolbar','figure');
plot(1:5); drawnow; 
 
% Enter plot-edit mode temporarily
plotedit(fig,'on'); drawnow
 
% Preserve the current mouse pointer location
oldPos = get(0,'PointerLocation');
 
% Move the mouse pointer to within the axes boundary
% ref: https://undocumentedmatlab.com/blog/undocumented-mouse-pointer-functions
figPos = getpixelposition(fig);   % figure position
axPos  = getpixelposition(gca,1); % axes position
figure(fig);  % ensure that the figure is in focus
newPos = figPos(1:2) + axPos(1:2) + axPos(3:4)/4;  % new pointer position
set(0,'PointerLocation',newPos);  % alternatives: moveptr(), java.awt.Robot.mouseMove()
 
% Simulate a right-click using Java robot
% ref: https://undocumentedmatlab.com/blog/gui-automation-robot
robot = java.awt.Robot;
robot.mousePress  (java.awt.event.InputEvent.BUTTON3_MASK); pause(0.1)
robot.mouseRelease(java.awt.event.InputEvent.BUTTON3_MASK); pause(0.1)
 
% Modify the <clear-axes> menu item
hMenuItem = findall(fig,'Label','Clear Axes');
if ~isempty(hMenuItem)
   label = '<html><b><i><font color="blue">Undocumented Matlab';
   callback = 'web(''https://undocumentedmatlab.com'',''-browser'');';
   set(hMenuItem, 'Label',label, 'Callback',callback);
end
 
% Hide the context menu by simulating a left-click slightly offset
set(0,'PointerLocation',newPos+[-2,2]);  % 2 pixels up-and-left
pause(0.1)
robot.mousePress  (java.awt.event.InputEvent.BUTTON1_MASK); pause(0.1)
robot.mouseRelease(java.awt.event.InputEvent.BUTTON1_MASK); pause(0.1)
 
% Exit plot-edit mode
plotedit(fig,'off'); drawnow
 
% Restore the mouse pointer to its previous location
set(0,'PointerLocation',oldPos);

In this code, I sprinkled a few pauses at several locations, to ensure that everything has time to fully render. Different pause values, or perhaps no pause at all, may be needed on your specific system.

Modifying the default context-menu shown in plot-edit mode may perhaps be an uncommon use-case. But the technique that I demonstrated above – of using a combination of Matlab and Java Robot commands to automate a certain animation – can well be used in many other use-cases where we cannot easily access the underlying code. For example, when the internal code is encoded/encrypted, or when a certain functionality (such as the plot-edit context-menu) is created on-the-fly.

If you have encountered a similar use-case where such automated animations can be used effectively, please add a comment below.

I was recently asked by a client to add a few buttons labeled “1”-“4” to a GUI toolbar. I thought: How hard could that be? Simply get the toolbar’s handle from the figure, then use the builtin uipushtool function to add a new button, specifying the label in the String property, right?

Well, not so fast it seems:

hToolbar = findall(hFig, 'tag','FigureToolBar');  % get the figure's toolbar handle
uipushtool(hToolbar, 'String','1');               % add a pushbutton to the toolbar
Error using uipushtool
There is no String property on the PushTool class. 

Apparently, for some unknown reason, standard Matlab only enables us to set the icon (CData) of a toolbar control, but not a text label.

Once again, Java to the rescue:

We first get the Java toolbar reference handle, then add the button in standard Matlab (using uipushtool, uitoggletool and their kin). We can now access the Java toolbar’s last component, relying on the fact that Matlab always adds new buttons at the end of the toolbar. Note that we need to use a short drawnow to ensure that the toolbar is fully re-rendered, otherwise we’d get an invalid Java handle. Finally, once we have this reference handle to the underlying Java button component, we can set and customize its label text and appearance (font face, border, size, alignment etc.):

hToolbar = findall(hFig, 'tag','FigureToolBar');     % get the figure's toolbar handle
jToolbar = hToolbar.JavaContainer.getComponentPeer;  % get the toolbar's Java handle
for buttonIdx = 1 : 4
    % First create the toolbar button using standard Matlab code
    label = num2str(buttonIdx);  % create a string label
    uipushtool(hToolbar, 'ClickedCallback',{@myCallback,analysisIdx}, 'TooltipString',['Run analysis #' label]);
 
    % Get the Java reference handle to the newly-created button
    drawnow; pause(0.01);  % allow the GUI time to re-render the toolbar
    jButton = jToolbar.getComponent(jToolbar.getComponentCount-1);
 
    % Set the button's label
    jButton.setText(label)
end

The standard Matlab toolbar button size (23×23 pixels) is too small to display more than a few characters. To display a longer label, we need to widen the button:

% Make the button wider than the standard 23 pixels
newSize = java.awt.Dimension(50, jButton.getHeight);
jButton.setMaximumSize(newSize)
jButton.setPreferredSize(newSize)
jButton.setSize(newSize)

Using a text label does not prevent us from also displaying an icon: In addition to the text label, we can also display a standard icon (by setting the button’s CData property in standard Matlab). This icon will be displayed to the left of the text label. You can widen the button, as shown in the code snippet above, to make space for both the icon and the label. If you want to move the label to a different location relative to the icon, simply modify the Java component’s HorizontalTextPosition property:

jButton.setHorizontalTextPosition(jButton.RIGHT);   % label right of icon (=default)
jButton.setHorizontalTextPosition(jButton.CENTER);  % label on top of icon
jButton.setHorizontalTextPosition(jButton.LEFT);    % label left of icon

In summary, here’s the code snippet that generated the screenshot above:

% Get the Matlab & Java handles to the figure's toolbar
hToolbar = findall(hFig, 'tag','FigureToolBar');     % get the figure's toolbar handle
jToolbar = hToolbar.JavaContainer.getComponentPeer;  % get the toolbar's Java handle
 
% Button #1: label only, no icon, 23x23 pixels
h1 = uipushtool(hToolbar);
drawnow; pause(0.01);
jButton = jToolbar.getComponent(jToolbar.getComponentCount-1);
jButton.setText('1')
 
% Create the icon CData from an icon file
graphIcon = fullfile(matlabroot,'/toolbox/matlab/icons/plotpicker-plot.gif');
[graphImg,map] = imread(graphIcon);
map(map(:,1)+map(:,2)+map(:,3)==3) = NaN;  % Convert white pixels => transparent background
cdata = ind2rgb(graphImg,map);
 
% Button #2: label centered on top of icon, 23x23 pixels
h2 = uipushtool(hToolbar, 'CData',cdata);
drawnow; pause(0.01);
jButton = jToolbar.getComponent(jToolbar.getComponentCount-1);
jButton.setText('2')
jButton.setHorizontalTextPosition(jButton.CENTER)
 
% Button #3: label on right of icon, 50x23 pixels
h3 = uipushtool(hToolbar, 'CData',cdata);
drawnow; pause(0.01);
jButton = jToolbar.getComponent(jToolbar.getComponentCount-1);
jButton.setText('3...')
d = java.awt.Dimension(50, jButton.getHeight);
jButton.setMaximumSize(d); jButton.setPreferredSize(d); jButton.setSize(d)
 
% Button #4: label on left of icon, 70x23 pixels
h4 = uipushtool(hToolbar, 'CData',cdata);
drawnow; pause(0.01);
jButton = jToolbar.getComponent(jToolbar.getComponentCount-1);
jButton.setText('and 4:')
jButton.setHorizontalTextPosition(jButton.LEFT)
d = java.awt.Dimension(70, jButton.getHeight);
jButton.setMaximumSize(d); jButton.setPreferredSize(d); jButton.setSize(d)

Many additional toolbar customizations can be found here and in my book “Undocumented Secrets of MATLAB-Java Programming“. If you’d like me to design a professional-looking GUI for you, please contact me.

Caveat emptor: all this only works with the regular Java-based GUI figures, not web-based (“App-Designer”) uifigures.

I would like to welcome back guest blogger Iliya Romm of Israel’s Technion Turbomachinery and Heat Transfer Laboratory. Today Iliya will discuss how to assign Matlab callbacks to JavaScript events in the new web-based uifigures. Other posts on customizations of web-based Matlab GUI can be found here.

On several occasions (including the previous post by Khris Griffis) I came across people who were really missing the ability to have Matlab respond to various JavaScript (JS) events. While MathWorks are working on their plans to incorporate something similar to this in future releases, we’ll explore the internal tools already available, in the hopes of finding a suitable intermediate solution.

Today I’d like to share a technique I’ve been experimenting with, allowing Matlab to respond to pretty much any JS event to which we can attach a listener. This is an overview of how it works:

1. create a UIFigure with the desired contents, and add to it (at least) one more dummy control, which has an associated Matlab callback.
2. execute a JS snippet that programmatically interacts with the dummy control, whenever some event-of-interest happens, causing the Matlab callback to fire.
3. query the webWindow, from within the Matlab callback, to retrieve any additional information (“payload”) that the JS passed.

This approach allows, for example, to easily respond to mouse events:

Consider the code below, which demonstrates different ways of responding to JS events. To run it, save the .m file function below (direct download link) and the four accompanying .js files in the same folder, then run jsEventDemo(demoNum), where demoNum is 1..4. Note: this code was developed on R2018a, unfortunately I cannot guarantee it works on other releases.

function varargout = jsEventDemo(demoNum)
   % Handle inputs and outputs
   if ~nargin
      demoNum = 4;
   end
   if ~nargout
      varargout = {};
   end
 
   % Create a simple figure:
   hFig = uifigure('Position',[680,680,330,240],'Resize','off');
   hTA = uitextarea(hFig, 'Value', 'Psst... Come here...!','Editable','off');
   [hWin,idTA] = mlapptools.getWebElements(hTA);
 
   % Open in browser (DEBUG):
   % mlapptools.waitForFigureReady(hFig); mlapptools.unlockUIFig(hFig); pause(1);
   % web(hWin.URL,'-browser')
 
   % Prepare the JS command corresponding to the requested demo (1-4)
   switch demoNum
      % Demo #1: Respond to mouse events, inside JS, using "onSomething" bindings:
      case 1
         % Example from: https://dojotoolkit.org/documentation/tutorials/1.10/events/#dom-events
         jsCommand = sprintf(fileread('jsDemo1.js'), idTA.ID_val);
 
      % Demo #2: Respond to mouse click events, inside JS, using pub/sub:
      case 2
         % Example from: https://dojotoolkit.org/documentation/tutorials/1.10/events/#publish-subscribe
         hTA.Value = 'Click here and see what happens';
         jsCommand = sprintf(fileread('jsDemo2.js'), idTA.ID_val);
 
      % Demo #3: Trigger Matlab callbacks programmatically from JS by "pressing" a fake button:
      case 3
         hB = uibutton(hFig, 'Visible', 'off', 'Position', [0 0 0 0], ...
                       'ButtonPushedFcn', @fakeButtonCallback);
         [~,idB] = mlapptools.getWebElements(hB);
         jsCommand = sprintf(fileread('jsDemo3.js'), idTA.ID_val, idB.ID_val);
 
      % Demo 4: Trigger Matlab callbacks and include a "payload" (i.e. eventData) JSON:
      case 4
         hB = uibutton(hFig, 'Visible', 'off', 'Position', [0 0 0 0],...
                      'ButtonPushedFcn', @(varargin)smartFakeCallback(varargin{:}, hWin));
         [~,idB] = mlapptools.getWebElements(hB);
         jsCommand = sprintf(fileread('jsDemo4.js'), idTA.ID_val, idB.ID_val);
   end % switch
 
   % Execute the JS command
   hWin.executeJS(jsCommand);
end
 
% Matlab callback function used by Demo #3
function fakeButtonCallback(obj, eventData) %#ok<INUSD>
   disp('Callback activated!');
   pause(2);
end
 
% Matlab callback function used by Demo #4
function smartFakeCallback(obj, eventData, hWin)
   % Retrieve and decode payload JSON:
   payload = jsondecode(hWin.executeJS('payload'));
 
   % Print payload summary to the command window:
   disp(['Responding to the fake ' eventData.EventName ...
         ' event with the payload: ' jsonencode(payload) '.']);
 
   % Update the TextArea
   switch char(payload.action)
      case 'enter',  act_txt = 'entered';
      case 'leave',  act_txt = 'left';
   end
   str = ["Mouse " + act_txt + ' from: '; ...
          "(" + payload.coord(1) + ',' + payload.coord(2) + ')'];  
   obj.Parent.Children(2).Value = str;
end

Several thoughts:

• The attached .js files will not work by themselves, rather, they require sprintf to replace the %s with valid widget IDs. Of course, these could be made into proper JS functions.
• Instead of loading the JS files using fileread, we could place the JS code directly in the jsCommand variable, as a Matlab string (char array)
• I tried getting it to work with a textarea control, so that we would get the payload right in the callback’s eventData object in Matlab, Unfortunately, I couldn’t get it to fire programmatically (solutions like this didn’t work). So instead, I store the payload as JSON, and retrieve it with jsondecode(hWin.executeJS('payload')) in the smartFakeCallback function.

JavaScript files

1. jsDemo1.js (direct download link):

   require(["dojo/on", "dojo/dom", "dojo/dom-style", "dojo/mouse"],
   	function(on, dom, domStyle, mouse) {
   		var myDiv = dom.byId("%s");
   		on(myDiv, mouse.enter, function(evt){
   			domStyle.set(myDiv, "backgroundColor", "red");
   		});
   		on(myDiv, mouse.leave, function(evt){
   			domStyle.set(myDiv, "backgroundColor", "");
   		});
   	});

2. jsDemo2.js (direct download link):

   require(["dojo/on", "dojo/topic", "dojo/dom"],
   	function(on, topic, dom) {
   		var myDiv = dom.byId("%s");
   		on(myDiv, "click", function() {
   			topic.publish("alertUser", "Your click was converted into an alert!");
   		});
   		topic.subscribe("alertUser", function(text){
   			alert(text);
   		});
   	});

3. jsDemo3.js (direct download link):

   require(["dojo/on", "dojo/dom", "dojo/dom-style", "dojo/mouse"],
   	function(on, dom, domStyle, mouse) {
   		var myDiv = dom.byId("%s");
   		var fakeButton = dom.byId("%s");
   		on(myDiv, mouse.enter, function(evt){
   			fakeButton.click();
   		});
   	});

4. jsDemo4.js (direct download link):

   var payload = [];
   require(["dojo/on", "dojo/dom", "dojo/topic", "dojo/mouse"],
   	function(on, dom, topic, mouse) {
   		var myDiv = dom.byId("%s");
   		var fakeButton = dom.byId("%s");
   		topic.subscribe("sendToMatlab", function(data){
   			payload = data;
   			fakeButton.click();
   		});
   		on(myDiv, mouse.enter, function(evt){
   			data = {action: "enter",
   				coord: [evt.clientX, evt.clientY]};
   			topic.publish("sendToMatlab", data);
   		});
   		on(myDiv, mouse.leave, function(evt){
   			data = {action: "leave",
   				coord: [evt.clientX, evt.clientY]};
   			topic.publish("sendToMatlab", data);
   		});
   	});

Conclusions

As you can see, this opens some interesting possibilities, and I encourage you to experiment with them yourself! This feature will likely be added to the mlapptools toolbox as soon as an intuitive API is conceived.

If you have any comments or questions about the code above, or just want to tell me how you harnessed this mechanism to upgrade your uifigure (I would love to hear about it!), feel free to leave a message below the gist on which this post is based (this way I get notifications!).

A while ago I posted the first of my planned miniseries on the Matlab toolstrip (ribbon). Today I will expand that post by discussing how toolstrips can be added to Matlab GUIs. This post will remain at a high-level as the previous post, with followup posts drilling into the technical details of the toolstrip components (inner packages and classes).

We can add a Matlab toolstrip to 3 types of Matlab GUI windows:

1. To a Java-based Matlab figure (so-called “legacy” figures, created using GUIDE or the figure function)
2. To a container window of docked Java-based figures, typically called an “App” (marketing name) or “Tool Group” (internal technical name)
3. To a JavaScript/HTML-based Matlab figure (so called “web” figures, created using App Designer or the uifigure function)

Today I will show how to add a basic dynamic toolstrip to a ToolGroup (App, window type #2):

ToolGroup with clients and dynamic toolstrip

Figure containers (“Tool Groups”)

Most Matlab users are familiar with window types #1 and #3 (legacy and web-based figures), but type #2 may seem strange. In fact, it shouldn’t be: All the Matlab “Apps” and Desktop components use such a container of docked clients. For example, both the Matlab Editor and Desktop are containers of individual client windows (individual files in the Editor; Command Window, Workspace etc. in the desktop).

Similarly, when we dock figures, they dock as client windows into a container called “Figures” (this can be controlled programmatically: see my setFigDockGroup utility on the File Exchange). This is the basis for all Matlab “Apps”, as far as I am aware (some Apps may possibly use a different GUI container, after all there are ~100 Matlab Apps and I’m not familiar with all of them). Such Apps are basically stand-alone Tool Groups (client container windows) that contain one or more docked figures, a toolstrip, and a side-panel with controls (so-called “Data Browser”).

Note: MathWorks uses confusing terminology here, using the same term “App” for both MathWorks-created GUIs containers (that have toolstrips, Data Browser and docked figures) and also user-created utilities on the File Exchange (that do not have these). Unfortunately, MathWorks has chosen not [yet] to release to the general public its set of tools that enable creating true “Apps”, i.e. those that have a toolstrip, Data Browser and docked figures.

Today’s post will attempt to fill this gap, by showing how we can create user Apps that have a toolstrip and docked figures. I will ignore the Data Browser today, and will describe it in a future post. Since docking figures into a standalone user-created container is a solved problem (using my setFigDockGroup utility), this post will focus on adding a toolstrip to such a container.

A ToolGroup object (matlab.ui.internal.desktop.ToolGroup) is created either implicitly (by docking a figure into a group that has a new name), or explicitly (by invoking its constructor):

% Create a new non-visible empty App (Tool Group)
hToolGroup = matlab.ui.internal.desktop.ToolGroup('Toolstrip example on UndocumentedMatlab.com');

Some things only work properly after the app is displayed, so let’s display the ToolGroup (however, note that for improved performance it is better to do whatever customizations and GUI updates that you can before the app is made visible):

% Display the ToolGroup window
hToolGroup.open();

Basic empty ToolGroup (without toolstrip or clients)

An annoying quirk with ToolGroups is that they automatically close when their reference handle is deleted from Matlab memory. The specific behavior changes depending on the contents of the container and the Matlab release, but in general it’s safest to preserve the hToolGroup variable, to prevent the window from closing, when this variable goes out of scope, when the function (in which we create the ToolGroup) returns. There are many ways to persist this variable. Here’s one alternative, in which we persist it in itself (or rather, attached to its internal Java peer control):

% Store toolgroup reference handle so that app will stay in memory
jToolGroup = hToolGroup.Peer;
internal.setJavaCustomData(jToolGroup, hToolGroup);

internal.setJavaCustomData is an undocumented Matlab function that adds a new custom property to a Java reference handle. In our case, it adds a CustomData property to the jToolGroup handle and sets its value to the Matlab hToolGroup handle. The source code for internal.setJavaCustomData is available in %matlabroot%/toolbox/shared/controllib/general/+internal/setJavaCustomData.m and is very simple: it essentially uses the old schema-based schema.prop method for adding new properties to handles. Schema is an old deprecated mechanism that is mostly replaced by the newer MCOS (Matlab Class Object System), but for some specific cases such as this it’s still very useful (the standard addprop function can add new properties to Matlab GUI handles, but not to Java reference handles).

Finally, let’s discard the Data Browser side panel (I’ll discuss it in a separate future post):

% Discard the Data-browser left panel
hToolGroup.disableDataBrowser();

Adding a toolstrip to the ToolGroup

Now that we have the basic container ready, let’s add a toolstrip. To simplify matters in this introductory post (after all, I have still not described the internal packages and classes that make up a toolstrip), we’ll use some of the tabs used in the showcaseToolGroup example that I discussed in my previous post. You can see the relevant source code in %matlabroot%/toolbox/matlab/toolstrip/+matlab/+ui/+internal/+desktop/*.m, in case you want to jump ahead and customize your own toolstrip tabs, groups and buttons. In the code snippet below, we first create an empty TabGroup, then add toolstrip tabs into it, and finally add this TabGroup into our ToolGroup using its addTabGroup(hTabGroup) method:

% Create a new tab group
%hTabGroup = matlab.ui.internal.desktop.showcaseBuildTabGroup('swing');
hTabGroup = matlab.ui.internal.toolstrip.TabGroup();
hTab1 = matlab.ui.internal.desktop.showcaseBuildTab_Buttons('swing');
hTabGroup.add(hTab1);
%hTabGroup.add(matlab.ui.internal.desktop.showcaseBuildTab_Gallery());
hTabGroup.add(matlab.ui.internal.desktop.showcaseBuildTab_Layout('swing'));
 
% Select tab #1 (common)
hTabGroup.SelectedTab = hTab1;
 
% Add the tab group to the built-in toolstrip
hToolGroup.addTabGroup(hTabGroup);

We now have an “App” that has a toolstrip, but no clients (yet), and a hidden Data Browser side-panel:

Now let’s add some clients (docked figures):

Adding clients (docked figures) to the ToolGroup

There are two easy variants for adding docked figures in a ToolGroup: The easiest is to use the ToolGroup’s addFigure(hFigure) method:

% Create a figure and dock it into the tool-group
hFig1 = figure('Name','3D');
surf(peaks);
hToolGroup.addFigure(hFig1);

The second variant enables to dock a figure that has a specific set of toolstrip tabs attached to it. These tabs will only display in the toolstrip when that particular figure has focus. We do this by creating a new TabGroup (just as we have done above), and then add the figure and TabGroup to the container using the ToolGroup’s addClientTabGroup(hFigure,hTabGroup) method:

% Create the 2nd figure
hFig2 = figure('Name','2D');
plot(rand(5));
 
% Add a few tabs that are only relevant to this specific figure
hTabGroup2 = TabGroup();
hTab2 = matlab.ui.internal.desktop.showcaseBuildTab_Selections();
hTabGroup2.add(hTab2);
hTabGroup2.add(matlab.ui.internal.desktop.showcaseBuildTab_EditValue());
 
% Add the figure and tabs to the ToolGroup
hToolGroup.addClientTabGroup(hFig2, hTabGroup2);

ToolGroup with clients and dynamic toolstrip

In this example, the “Selection” and “Values” toolstrip tabs only appear when the 2nd figure (“2D”) has focus. A similar behavior exists in the Matlab Desktop and Editor, where some tabs are only shown when certain clients have focus.

Removing the View tab

Note that the “View” toolstrip tab (which enables setting the appearance of the docked figures: layout, tab positions (top/bottom/left/right), ordering etc.) is automatically added to the toolstrip and always appears last. We can remove this View tab using the ToolGroup’s hideViewTab() method. The tab will not immediately be removed, only when the toolstrip is repainted, for example, when we switch focus between the docked figures:

hToolGroup.hideViewTab;  % toolstrip View tab is still visible at this point
figure(hFig1);  % change focus to hFig1 - toolstrip is repainted without View tab

Conclusion

It’s relatively easy to dock figures into a standalone “App” window that has a custom toolstrip, which can even be dynamically modified based on the figure which is currently in focus. Naturally, this has little benefit if we cannot customize the toolstrip components: labels, icons, control type, grouping and most importantly – callbacks. This topic deserves a dedicated post, which I plan to be the next in this miniseries. Stay tuned – hopefully the next post will not take me as long to publish as this post (I was quite busy recently)…

In the previous post I showed how we can create custom Matlab apps. In such apps, the toolstrip is very often an important part. Today I continue my miniseries on toolstrips. Toolstrips can be a bit complex so I’m trying to proceed slowly, with each post in the miniseries building on the previous posts. So I encourage you to review the earlier posts in the miniseries (part1, part2) before reading this post.

A Matlab toolstrip is composed of a hierarchy of user-interface objects as follows (all objects are classes within the matlab.ui.internal.toolstrip package):

Anatomy of a Matlab app with toolstrip

• TabGroup
  • Tab
    • Section
      • Column
        • Component
        • Component
        • …
      • Column
      • …
    • Section
    • …
  • Tab
  • …
• TabGroup
• …

In this post I explain how we can create a custom toolstrip that contains tabs, sections, and basic controls that interact with the user and the docked figures. The following posts will show more advanced customizations and more complex controls, as well as showing alternative ways of creating the toolstrip.

1. Creating a bare toolstrip and new tabs

We start with a new ToolGroup that has a bare toolstrip and a docked figure (for details and explanations refer to the previous post):

% Create a new ToolGroup ("app") with a hidden DataBrowser
hToolGroup = matlab.ui.internal.desktop.ToolGroup('Toolstrip example on UndocumentedMatlab.com');
hToolGroup.disableDataBrowser();
hToolGroup.open();  % this may be postponed further down for improved performance
 
% Store toolgroup reference handle so that app will stay in memory
jToolGroup = hToolGroup.Peer;
internal.setJavaCustomData(jToolGroup, hToolGroup);
 
% Create two figures and dock them into the ToolGroup
hFig1 = figure('Name','3D');  surf(peaks);
hToolGroup.addFigure(hFig1);

We now create a new TabGroup and and it to our ToolGroup:

import matlab.ui.internal.toolstrip.*  % for convenience below
hTabGroup = TabGroup();
hToolGroup.addTabGroup(hTabGroup);

We can add a new Tab to the TabGroup using either of two methods:

1. Create a new Tab object and then use TabGroup.add(hTab,index) to add it to a parent TabGroup. The index argument is optional – if specified the section is inserted at that index location; if not, it is added at the end of the tab-group. Sample usage:

   hTab = Tab('Data');
   hTabGroup.add(hTab);  % add to tab as the last section
   hTabGroup.add(hTab,3);  % add to tab as the 3rd section

2. Call TabGroup.addTab(title). This creates a new tab with the specified title (default: ”) and adds it at the end of the tab-group. The new tab’s handle is returned by the function. Sample usage:

   hTabGroup.addTab('Data');  % add to tab-group as the last tab

This creates an empty “Data” tab in our app toolstrip. Note that the tab title is capitalized (“DATA”), despite the fact that we set its Title property to 'Data'. Also note that while the tab’s Title property can be updated after the tab is created, in practice the tab title does not seem to change.

Lastly, note that a “VIEW” tab is automatically added to our toolstrip. As explained in the previous post, we can remove it using hToolGroup.hideViewTab; (refer to the previous post for details).

2. Adding sections to a toolstrip tab

Each toolstrip Tab is composed of Sections, that holds the actual components. We cannot add components directly to a Tab: they have to be contained within a Section. A toolstrip Tab can only contain Sections as direct children.

We can add a new section to a Tab using either of two methods, in a similar way to the that way we added a new tab above:

1. Create a new Section object and then use Tab.add(hSection,index) to add it to a parent Tab. The index argument is optional – if specified the section is inserted at that index location; if not, it is added at the end of the tab. Sample usage:

   hSection = Section('Section title');
   hTab.add(hSection);  % add to tab as the last section
   hTab.add(hSection,3);  % add to tab as the 3rd section

2. Call Tab.addSection(title). This creates a new section with the specified title (default: ”) and adds it at the end of the tab. The new section’s handle is returned by the function. Sample usage:

   hTab.addSection('Section title');  % add to tab as the last section

Note that the help section for Tab.addSection() indicates that it’s possible to specify 2 string input args (presumably Title and Tag), but this is in fact wrong and causes a run-time error, since Section constructor only accepts a single argument (Title), at least as of R2018b.

The Section‘s Title property can be set both in the constructor, as well as updated later. In addition, we can also set the Tag and CollapsePriority properties after the section object is created (these properties cannot be set in the constructor call):

hSection.Title = 'New title';    % can also be set in constructor call
hSection.Tag = 'section #1';     % cannot be set in constructor call
hSection.CollapsePriority = 10;  % cannot be set in constructor call

The CollapsePriority property is responsible for controlling the order in which sections and their internal components collapse into a drop-down when the window is resized to a smaller width.

Like tabs, section titles also appear capitalized. However, unlike the section titles can indeed be modified in run-time.

3. Adding columns to a tab section

Each Section in a toolstrip Tab is composed of Columns, and each Column can contain 1-3 Components. This is a very effective layout for toolstrip controls that answers the vast majority of use-cases. In some special cases we might need more flexibility with the component layout within a Tab – I will explain this in a future post. But for now let’s stick to the standard Tab-Section-Column-Component framework.

We can add columns to a section using (guess what?) either of two methods, as above:

1. Create a new Column object and then use Section.add(hColumn,index) to add it to a parent Section. The index argument is optional – if specified the column is inserted at that index location; if not, it is added at the end of the section. Sample usage:

   hColumn = Column('HorizontalAlignment','center', 'Width',150);
   hSection.add(hColumn);  % add to section as the last column
   hSection.add(hColumn,3);  % add to section as the 3rd column

2. Call Tab.addSection(title). This creates a new section with the specified title (default: ”) and adds it at the end of the tab. The new section’s handle is returned by the function. Sample usage:

   hSection.addColumn('HorizontalAlignment','center', 'Width',150);  % add to section as the last column

We can set the Column‘s HorizontalAlignment and Width properties only in the constructor call, not later via direct assignments. In contrast, the Tag property cannot be set in the constructor, only via direct assignment:

hColumn.HorizontalAlignment = 'right';  % error: can only be set via constructor call: Column('HorizontalAlignment','right', ...)
hColumn.Width = 150;                    % error: can only be set via constructor call: Column('Width',150, ...)
hColumn.Tag = 'column #2';              % ok: cannot be set via the constructor call!

This is indeed confusing and non-intuitive. Perhaps this is part of the reason that the toolstrip API is still not considered stable enough for a general documented release.

4. Adding controls to a section column

Each section column contains 1 or more Components. These can be push/toggle/split/radio buttons, checkboxes, drop-downs, sliders, spinners, lists etc. Take a look at matlabroot%/toolbox/matlab/toolstrip/+matlab/+ui/+internal/+toolstrip/ for a full list of available controls. I’ll discuss a few basic controls in this post, and more complex ones in future posts.

As above, there are two methods for adding components to a section column, but they have different purposes:

1. Column.addEmptyControl() adds a filler space in the next position of the column. This is used to display the colorbar control at the center of the column in the usage snippet below.
2. Create a new Component object and then use Column.add(hComponent, index) to add it to a parent Column. The index argument is optional – if specified the component is inserted at that index location; if not, it is added at the end of the column. Sample usage:

   hButton = Button('Click me!');
   hColumn.add(hButton);  % add to column as the last component
   hColumn.add(hButton,3);  % add to column as the 3rd component

Component objects (matlab.ui.internal.toolstrip.base.Component, which all usable toolstrip controls inherit) have several common properties. Leaving aside the more complex components for now, most usable controls include the following properties:

• Text – text label, displayed next to the control icon (pity that MathWorks didn’t call this property String or Label, in line with uicontrols/menu-items)
• Description – tooltip, displayed when hovering the mouse over the control (pity that MathWorks didn’t call this property Tooltip in line with uicontrols/menu-items)
• Tag – a string, as all other Matlab HG objects. Controls are searchable by their Tag via their container’s find(tag) and findAll(tag) methods (again, I don’t quite understand why not use findobj and findall as with the rest of Matlab HG…).
• Enabled – a logical value (true/false), true by default
• Icon – the icon used next to the Text label. We can use the Icon constructor (that expects the full path of a PNG/JPG file), or one of its static icons (e.g. Icon.REFRESH_16). Icons will be discussed in detail in the following post; in the meantime you can see various usage examples below.

Each control also has one or more callbacks that can be specified, as settable properties and/or as events that can be listened-to using the addlistener function. This too will be discussed in detail in the next post, but in the meantime you can see various usage examples below.

Columns can have 1-3 components:

• If only 1 component is specified, it is allocated the full column height, effectively creating a large control, with the Icon on top (typically a 24×24 icon) and the Text label beneath.
• If 2 or 3 components are specified, then smaller controls are displayed, with the Text label to the right of the Icon (typically 16×16), and the controls evenly spaced within the column.
• If you try to add more than 3 components to a Column, you’ll get a run-time error.

5. Usage example

Here is a short usage example showing the above concepts. The code is not pretty by any means – I intentionally wanted to display multiple different ways of adding components, specifying properties and callbacks etc. It is meant merely as an educational tool, and is not close to being ready for production code. So please don’t complain about the known fact that the code is ugly, non-robust, and in general exhibits bad programming practices. The following code snippets assume that you have already ran the code in paragraph 1 above:

Push-buttons section (3 columns)

Toolstrip example (basic controls)

section1 = hTab.addSection('Push buttons');
 
column1a = section1.addColumn();
icon = Icon.REFRESH_24; % built-in: see Icon.showStandardIcons()
button = Button('Refresh all',icon);
button.Description = 'Refresh the charted data - all axes';
button.ButtonPushedFcn = @refreshAllData;
column1a.add(button);
function refreshAllData(hAction,hEventData)
    hAxes = gca;
    hChildren = hAxes.Children;
    for idx = 1 : numel(hChildren)
        hChild = hChildren(idx);
        hChild.XData = -hChild.XData;
        hChild.YData = -hChild.YData;
        hChild.ZData = -hChild.ZData;
    end
end
 
column1b = section1.addColumn();
addRefresh2Button('X','Y');
addRefresh2Button('Y','Z');
function addRefresh2Button(type1, type2)
    import matlab.ui.internal.toolstrip.*
    hButton = Button(['Refresh ' type1 ',' type2], Icon.RESTORE_16);
    hButton.Description = ['Refresh the charted data - ' type1 ',' type2 ' axes'];
    hButton.ButtonPushedFcn = {@refres2AxisData, type1, type2};
    column1b.add(hButton);
 
    function refres2AxisData(~,~,type1,type2)
        hAxes = gca;
        hChildren = hAxes.Children;
        for idx = 1 : numel(hChildren)
            hChild = hChildren(idx);
            hChild.([type1 'Data']) = -hChild.([type1 'Data']);
            hChild.([type2 'Data']) = -hChild.([type2 'Data']);
        end
    end
end
 
column1c = section1.addColumn();
addRefresh1Button('X');
addRefresh1Button('Y');
addRefresh1Button('Z');
function addRefresh1Button(type)
    import matlab.ui.internal.toolstrip.*
    hButton = Button(['Refresh ' type], Icon.REDO_16);
    hButton.Description = ['Refresh the charted data - ' type ' axes'];
    addlistener(hButton, 'ButtonPushed', @refres1AxisData);  % {} not supported!
    column1c.add(hButton);
 
    function refres1AxisData(h,e)
        hAxes = gca;
        hChildren = hAxes.Children;
        for idx = 1 : numel(hChildren)
            hChild = hChildren(idx);
            hChild.([type 'Data']) = -hChild.([type 'Data']);
        end
    end
end

Toggle buttons section (2 columns)

section2 = hTab.addSection('Toggle buttons');
section2.CollapsePriority = 2;
 
column1 = Column();
section2.add(column1);
%icon = Icon.LEGEND_24;
icon = Icon(fullfile(matlabroot,'toolbox','shared','controllib','general','resources','toolstrip_icons','Legend_24.png')); % PNG/JPG image file (not GIF!)
button = ToggleButton('Legend',icon);
button.Description = 'Toggle legend display';
addlistener(button, 'ValueChanged', @(h,e)legend('toggle'));
column1.add(button);
 
column2 = section2.addColumn();
imagefile = fullfile(matlabroot,'toolbox','matlab','icons','tool_colorbar.png');
jIcon = javax.swing.ImageIcon(imagefile); % Java ImageIcon from file (inc. GIF)
%jIcon = javax.swing.ImageIcon(jIcon.getImage.getScaledInstance(24,24,jIcon.getImage.SCALE_SMOOTH))  % Resize icon to 24x24
icon = Icon(jIcon);
button = ToggleButton('Colorbar',icon);
button.Description = 'Toggle colorbar display';
button.ValueChangedFcn = @toggleColorbar;
column2.addEmptyControl();
column2.add(button);
column2.addEmptyControl();
function toggleColorbar(hAction,hEventData)
    if hAction.Selected
        colorbar;
    else
        colorbar('off');
    end
end

Checkboxes section (1 column 150px-wide), placed after the push-buttons section

section3 = Section('Checkboxes');
section3.CollapsePriority = 1;
hTab.add(section3, 2);
 
column3 = section3.addColumn('HorizontalAlignment','left', 'Width',150);
 
button = CheckBox('Axes borders', true);
button.ValueChangedFcn = @toggleAxes;
button.Description = 'Axes borders';
column3.add(button);
function toggleAxes(hAction,~)
    if hAction.Value
        set(gca,'Visible','on');
    else
        set(gca,'Visible','off');
    end
end
 
button = CheckBox('Log scaling', false);
button.addlistener('ValueChanged',@toggleLogY);
button.Description = 'Log scaling';
column3.add(button);
function toggleLogY(hAction,~)
    if hAction.Value, type = 'log'; else, type = 'linear'; end
    set(gca, 'XScale',type, 'YScale',type, 'ZScale',type);
end
 
button = CheckBox('Inverted Y', false);
button.addlistener('ValueChanged',@toggleInvY);
button.Description = 'Invert Y axis';
column3.add(button);
function toggleInvY(hAction,~)
    if hAction.Value, type = 'reverse'; else, type = 'normal'; end
    set(gca, 'YDir',type);
end

Summary

Creating a custom app toolstrip requires careful planning of the tabs, sections, controls and their layout, as well as preparation of the icons, labels and callbacks. Once you start playing with the toolstrip API, you’ll see that it’s quite easy to understand and to use. I think MathWorks did a good job in general with this API, and it’s a pity that they did not make it public or official long ago (the MCOS API discussed above existed since 2014-2015; earlier versions existed at least as far back as 2011). Comparing the changes made in the API between R2018a and R2018b shows quite minor differences, which may possibly means that the API is now considered stable, and therefore that it might well be made public in some near-term future. Still, note that this API may well change in future releases (for example, naming of the control properties that I mentioned above). It works well in R2018b, as well as in the past several Matlab releases, but this could well change in the future, so beware.

In the following posts I will discuss advanced control customizations (icons, callbacks, collapsibility etc.), complex controls (drop-downs, pop-ups, lists, button groups, items gallery etc.) and low-level toolstrip creation and customization. As I said above, Matlab toolstrips are quite an extensive subject and so I plan to proceed slowly, with each post building on its predecessors. Stay tuned!

In the meantime, if you would like me to assist you in building a custom toolstrip or GUI for your Matlab program, please let me know.

I planned to post a new article in my toolstrip mini-series, but then I came across something that I believe has a much greater importance and impacts many more Matlab users: the change in Matlab R2018b’s figure toolbar, where the axes controls (zoom, pan, rotate etc.) were moved to be next to the axes, which remain hidden until you move your mouse over the axes. Many users have complained about this unexpected change in the user interface of such important data exploration functionality:

R2018a (standard toolbar)

R2018b (integrated axes toolbar)

Luckily, we can revert the change, as was recently explained in this Answers thread:

addToolbarExplorationButtons(gcf) % Add the axes controls back to the figure toolbar
 
hAxes.Toolbar.Visible = 'off'; % Hide the integrated axes toolbar
%or:
hAxes.Toolbar = []; % Remove the axes toolbar data

And if you want to make these changes permanent (in other words, so that they would happen automatically whenever you open a new figure or create a new axes), then add the following code snippet to your startup.m file (in your Matlab startup folder):

try %#ok
    if ~verLessThan('matlab','9.5')
        set(groot,'defaultFigureCreateFcn',@(fig,~)addToolbarExplorationButtons(fig));
        set(groot,'defaultAxesCreateFcn',  @(ax,~)set(ax.Toolbar,'Visible','off'));
    end
end

MathWorks is taking a lot of heat over this change, and I agree that it could have done a better job of communicating the workaround in placing it as settable configurations in the Preferences panel or elsewhere. Whenever an existing functionality is broken, certainly one as critical as the basic data-exploration controls, MathWorks should take extra care to enable and communicate workarounds and settable configurations that would enable users a gradual smooth transition. Having said this, MathWorks does communicate the workaround in its release notes (I’m not sure whether this was there from the very beginning or only recently added, but it’s there now).

In my opinion the change was *not* driven by the marketing guys (as was the Desktop change from toolbars to toolstrip back in 2012 which received similar backlash, and despite the heated accusations in the above-mentioned Answers thread). Instead, I believe that this change was technically-driven, as part of MathWorks’ ongoing infrastructure changes to make Matlab increasingly web-friendly. The goal is that eventually all the figure functionality could transition to Java-script -based uifigures, replacing the current (“legacy”) Java-based figures, and enabling Matlab to work remotely, via any browser-enabled device (mobiles included), and not be tied to desktop operating systems. In this respect, toolbars do not transition well to webpages/Javascript, but the integrated axes toolbar does. Like it or not, eventually all of Matlab’s figures will become web-enabled content, and this is simply one step in this long journey. There will surely be other painful steps along the way, but hopefully MathWorks would learn a lesson from this change, and would make the transition smoother in the future.

Once you regain your composure and take the context into consideration, you might wish to let MathWorks know what you think of the toolbar redesign here. Please don’t complain to me – I’m only the messenger…

Merry Christmas everybody!

p.s. Once of the complaints against the new axes toolbar is that it hurts productivity by forcing users to wait for the toolbar to fade-in and become clickable. Apparently the axes toolbar has a hidden private property called FadeGroup that presumably controls the fade-in/out effect. This can be accessed as follows:

hFadeGroup = struct(hAxes.Toolbar).FadeGroup  % hAxes is the axes handle

I have not [yet] discovered if and how this object can be customized to remove the fade animation or control its duration, but perhaps some smart hack would discover and post the workaround here (or let me know in a private message that I would then publish anonymously).