Using coremltools in a CI/CD pipeline

Question

Created 2w

Replies 1

Boosts 0

Participants 2

Hi everyone 👋

I'd like to use coremltools to see how well a model performs on a remote device as part of a CI/CD pipeline. According to the Core ML Tools "Debugging and Performance Utilities" guide, remote devices must be in a "connected" state in order for coremltools to install the ModelRunner application.

The devices in our system have a "paired" state, and I'm unable to set the them as "connected." The only way I know how to connect a device is to physically plug it in to a computer and open Xcode. I don't have physical access to the devices in the CI/CD system, and the host computer that interacts with them doesn't have Xcode installed.

Here are some questions I've been looking into and would love some help answering:

Has anyone managed to use the coremltools performance utilities in a similar system?
Can you put a device in a "connected" state if you don't have physical access to the device and if you only have access to Xcode command line tools and not the Xcode app?
Is it at all possible to install the coremltools ModelRunner application on a "paired" device, for example, by manually building the app and installing it with devicectl? Would other utilities, such as the MLModelBenchmarker work as expected if the app is installed this way?

Thank you!

Boost

Answer 1

Engineer OP

Apple

1w

Hello, if the device doesn’t appear as “Connected”, it usually indicates that the device is discoverable but the transport connection hasn’t been established. coremltools requires the transport layer to be set up in order to communicate with the ModelRunner application.

Try using devicectl to install and launch a sample application. If that works, the issue is likely related to the device state. In that case, please file an issue here: https://github.com/apple/coremltools/issues

0