Posts under Machine Learning & AI topic

Post

Replies

Boosts

Views

Activity

WebKit interop with AppIntents
Is there any new functionality in WebKit for bridging WebViews with the AppIntents framework? For example, is there a simple way to create an IndexedEntity whose content is provided by a WKWebView? Also, if I use appEntityIdentifer in a View that contains a WKWebView, will Siri see the web view’s content?
2
1
62
6d
Adopting AppEntity schemas
I'm trying to get my head around adopting AppEntity schemas. Is it OK to shoehorn a closely-related concept from our apps into AppEntity schemas, even if they'r not a perfect fit? For e.g. I have 'tasks' in my app, that are related to a 'person' entity. Would it work to adopt a schema like @AppEntity(schema: .reminders.reminder) struct TaskEntity: IndexedEntity { var personForTask: PersonEntity ... fill in other required fields } Most of the fields are optional or arrays, so it could work, but one of the required fields seems to be var list: ListEntity, so I would have to conform to that in some way. Similarly, I would want to adopt the .reminders.createReminder @AppIntent for my app, even though I won't be able to support all the parameters that it can support. Would that be OK as well?
1
0
62
6d
model variants with Siri AI as it relates to App Intents
The Keynote/SOTU mentioned that phone hardware wise, only iPhone Air and 17 Pro are getting the highest quality local models. Does this only impact 1st party system features, or does that impact us as well? In other words, will our schema conforming App Intents behave differently on this subset of devices. Or is testing on any Apple Intelligence supported hardware sufficient?
1
0
62
6d
How much of the new Siri can a third-party app access beyond App Intents?
Hi all. I'm building a chat assistant app and want to bring in the new Siri. I can adopt App Intents, but I'm trying to understand how much of the new Siri's functionality, tools, and system access I actually get as a third-party developer. Is App Intents the full extent of what's available to third-party apps, or is more of the new Siri's capabilities and system access exposed to us? Thanks.
0
0
63
6d
Questions for App Intents & Siri Q&A
Hello, I have a few questions for the App Intents & Siri Q&A Session. Can I make functionality of my app available to Siri AI if they are NOT from one of the predefined App Schema domains? (This was already asked here: https://developer.apple.com/forums/thread/829586 ) Can I define my own App Schemas & App Schema Domains? Is it possible to hand over from Siri to the App providing the App Intent (i.e. open the app from a Siri as result of an action)? Will the processing of an Intent result in an App Lifecycle change? How is compute time & network access managed when processing the intent? E.g., would I be able to make backend requests if needed?
2
1
194
6d
Testing App Shortcuts on simulator
Hi, In the simulator, when I use the Shortcuts app to get to my list of pre-defined App Shortcuts list, tapping on any of the shortcuts gives me an alert "Unable to run App Shortcut". This is also try in the new iOS27 DeviceHub. It runs fine on device, and on the simulator if I create a new shortcut and use the same action. Is this expected behavior?
1
0
77
6d
integration into my music app
My app is built around music albums. How would I support the utterance "open this in Apple Music" when a user is looking at an album in my app? Is a .userActivity with an associated appEntityIdentifier enough? If my AppEntity had the Apple Music ID in one of it's properties would the system figure it out? Or would I need a transfer representation too? This introduction of IntentValueRepresentation was confusing for me. It seems from the talk that maybe this is required to make this work, but doesn't seem like there is any relevant _IntentValue for my use case.
1
0
59
6d
multi-type transferRepresentation
Say you have an App Entity representing an image. And an app intent that returns it as a value. Is there any way to define a compound transfer representation? For example, if the user wanted to create a Shortcut that sent a message or created a note with the image along with the image title as plain text. Last I attempted this, the system rendered a single compound image with the title baked in underneath the actual image, as opposed to the image followed by a text string underneath it.
1
0
42
6d
Structuring Multi-Step Confirmations and Disambiguation in Siri Voice UX
We want to support voice workflows that require safety confirmations (e.g., 'Siri, prune my stale database caches' followed by Siri asking 'This will delete 42 entries, are you sure you want to proceed?'). How do we cleanly model multi-turn dialogs in Swift using AppIntent without forcing the user to re-trigger Siri? What is the correct way to throw dialog prompts or handle dynamic parameter disambiguation (e.g., if the user says 'Siri, check status' and multiple daemon instances are running, how do we return a list of choices to Siri to read out loud)?
1
0
37
6d
Use ShowInAppSearchResultsIntent with custom Parameters
I’m currently using ShowInAppSearchResultsIntent to open the app in a selectable view with the search results. The user can choose which view to pick via a @Parameter. Though with iOS 27 I can’t compile this intent anymore, because of this error: 'ShowInAppSearchResultsIntent' must only have a 'criteria' parameter What is the best practice to offer the same selection of the search view with iOS 27 and newer then? My Code import AppIntents @AppIntent(schema: .system.search) struct SearchAppIntent: ShowInAppSearchResultsIntent { static let searchScopes: [StringSearchScope] = [.general] var criteria: StringSearchCriteria // MARK: Parameters @Parameter(default: .loans) var target: SearchView? // MARK: Action Text static var parameterSummary: some ParameterSummary { Summary("Search \(\.$criteria) in \(\.$target)", table: "Shortcuts") } // MARK: Action @MainActor func perform() async throws -> some IntentResult { switch target { case .general, nil: // Open app search tab NavigationManager.shared.openAppSearch(with: criteria.term) case .contacts: // Open contacts tab NavigationManager.shared.openContactsSearch(with: criteria.term) } return .result() } }
2
0
62
6d
Handling Long-Running Background Actions and Network Sync in App Intents
Some of our intents involve initiating local-to-remote database synchronization (e.g., syncing local memory metrics to a remote MongoDB instance). This operation can take between 5 to 15 seconds depending on network conditions. What are the strict execution timeout limits for App Intents running in the background on macOS and iOS (especially when triggered via Siri without active UI)? If the operation exceeds the default limit, will Siri forcefully terminate the intent? How can we request background execution extension or hand off the work to a background download session/daemon while reporting progress back to Siri in real-time? Does the system support asynchronous yielding (Progress) so Siri can display a progress bar or status updates (e.g., 'Syncing 50%... Done') during execution?
1
0
51
6d
Intents for Live Activity Buttons/Toggles take a while to execute
Apple Music's Live Activity has instantly responsive prev/next/pause intents. Similar Buttons and Toggles on our Live Activity shimmer for a second or two as the LiveActivityIntent perform body executes light work in our main process, the content update is submitted and serialized, and that update finally renders. That makes our Live Activity a bit clunky. The delays also nixed some useful interactive feature ideas. The slow response is true even if our main process was running in the background with an active HKLiveWorkoutBuilder. Is this an expected limitation? Is there a strategy we should have taken to avoid this? (We do use optimistic animations via Toggle to soften the edges a bit.)
2
0
52
6d
Handling Raw User Input Prompts (Open-ended Dictation)
In some of our intents, we want the user to dictate a custom prompt or text body to be stored in the memory cache (e.g., 'Siri, add memory tag with content: Remember to invalid auth caches when schema bumps'). How can we define an AppIntent parameter that accepts open-ended dictation or arbitrary string blocks from Siri's speech-to-text translator? Are there length limits or language restrictions on string parameters parsed by Siri?
1
0
49
6d
Flexibility in adoption AppEntity Schemas
Hi, I have a list of contacts that I store in my app. Can I list all the contacts and their properties into the new Siri by adopting this schema? @available(iOS 27.0, *) @AppEntity(schema: .messages.messagePerson) struct CJPersonAppIndexedEntity: IndexedEntity { ... } It's not really part of the 'messages' domain, but is it correct to use? Also, if I have a concept of a 'task' and 'file' in my app? Can I use the .reminders domain and use .createreminder intent, and mix that 'reminders' and 'files' domains with the 'messages' domain, all in the same app?
1
0
43
6d
How to show correct title for intents in Spotlight
In my plant watering app, I created a PlantEntity and a WaterPlantIntent that edits the plant to mark it as watered. PlantEntity conforms to AppEntity and IndexedEntity. I also created OpenPlantIntent that conforms to OpenIntent. My AppShortcutsProvider is implemented like so: struct ShortcutsProvider: AppShortcutsProvider { static var appShortcuts: [AppShortcut] { return [ AppShortcut( intent: WaterPlantIntent(), phrases: [ "Water plant in \(.applicationName)", "Water \(\.$plantEntity) in \(.applicationName)" ], shortTitle: "Water Plant", systemImageName: "drop.fill" ) ] } } Now, when I search in Spotlight for a plant name, two results are shown but both are titled the name of the plant. The one in the App Name section just opens the plant in the app but the one in the Top Hit section actually performs the WaterPlantIntent! There's no indication that would occur. I expected it to be titled Water Plant Name so users know what shortcut is being run. Is there a way I can configure my code to show the proper title here?
3
0
73
6d
Will Siri AI only use AppIntents that have AppSchemas when "vibe coding" shortcuts?
I have an app that allows users to create charts and add series to them using AppIntents in Shortcuts. I couldn't find any AppSchemas that would match my entities in the official documentation. While "vibe coding" shortcuts, will Siri AI only access app intents through AppSchemas? If 1 is true, is there any alternative to AppSchemas to make AppIntents discoverable by Siri?
1
0
108
6d
WebKit interop with AppIntents
Is there any new functionality in WebKit for bridging WebViews with the AppIntents framework? For example, is there a simple way to create an IndexedEntity whose content is provided by a WKWebView? Also, if I use appEntityIdentifer in a View that contains a WKWebView, will Siri see the web view’s content?
Replies
2
Boosts
1
Views
62
Activity
6d
Adopting AppEntity schemas
I'm trying to get my head around adopting AppEntity schemas. Is it OK to shoehorn a closely-related concept from our apps into AppEntity schemas, even if they'r not a perfect fit? For e.g. I have 'tasks' in my app, that are related to a 'person' entity. Would it work to adopt a schema like @AppEntity(schema: .reminders.reminder) struct TaskEntity: IndexedEntity { var personForTask: PersonEntity ... fill in other required fields } Most of the fields are optional or arrays, so it could work, but one of the required fields seems to be var list: ListEntity, so I would have to conform to that in some way. Similarly, I would want to adopt the .reminders.createReminder @AppIntent for my app, even though I won't be able to support all the parameters that it can support. Would that be OK as well?
Replies
1
Boosts
0
Views
62
Activity
6d
Limit for indexed AppEntity instances?
Is there a hard limitation on the size or total number of entities that can be indexed by a single application? Additionally, are there any recommended performance thresholds or best practices when dealing with a large volume of entities (e.g., thousands to ~30K)?
Replies
1
Boosts
0
Views
119
Activity
6d
model variants with Siri AI as it relates to App Intents
The Keynote/SOTU mentioned that phone hardware wise, only iPhone Air and 17 Pro are getting the highest quality local models. Does this only impact 1st party system features, or does that impact us as well? In other words, will our schema conforming App Intents behave differently on this subset of devices. Or is testing on any Apple Intelligence supported hardware sufficient?
Replies
1
Boosts
0
Views
62
Activity
6d
How much of the new Siri can a third-party app access beyond App Intents?
Hi all. I'm building a chat assistant app and want to bring in the new Siri. I can adopt App Intents, but I'm trying to understand how much of the new Siri's functionality, tools, and system access I actually get as a third-party developer. Is App Intents the full extent of what's available to third-party apps, or is more of the new Siri's capabilities and system access exposed to us? Thanks.
Replies
0
Boosts
0
Views
63
Activity
6d
Questions for App Intents & Siri Q&A
Hello, I have a few questions for the App Intents & Siri Q&A Session. Can I make functionality of my app available to Siri AI if they are NOT from one of the predefined App Schema domains? (This was already asked here: https://developer.apple.com/forums/thread/829586 ) Can I define my own App Schemas & App Schema Domains? Is it possible to hand over from Siri to the App providing the App Intent (i.e. open the app from a Siri as result of an action)? Will the processing of an Intent result in an App Lifecycle change? How is compute time & network access managed when processing the intent? E.g., would I be able to make backend requests if needed?
Replies
2
Boosts
1
Views
194
Activity
6d
Testing App Shortcuts on simulator
Hi, In the simulator, when I use the Shortcuts app to get to my list of pre-defined App Shortcuts list, tapping on any of the shortcuts gives me an alert "Unable to run App Shortcut". This is also try in the new iOS27 DeviceHub. It runs fine on device, and on the simulator if I create a new shortcut and use the same action. Is this expected behavior?
Replies
1
Boosts
0
Views
77
Activity
6d
integration into my music app
My app is built around music albums. How would I support the utterance "open this in Apple Music" when a user is looking at an album in my app? Is a .userActivity with an associated appEntityIdentifier enough? If my AppEntity had the Apple Music ID in one of it's properties would the system figure it out? Or would I need a transfer representation too? This introduction of IntentValueRepresentation was confusing for me. It seems from the talk that maybe this is required to make this work, but doesn't seem like there is any relevant _IntentValue for my use case.
Replies
1
Boosts
0
Views
59
Activity
6d
multi-type transferRepresentation
Say you have an App Entity representing an image. And an app intent that returns it as a value. Is there any way to define a compound transfer representation? For example, if the user wanted to create a Shortcut that sent a message or created a note with the image along with the image title as plain text. Last I attempted this, the system rendered a single compound image with the title baked in underneath the actual image, as opposed to the image followed by a text string underneath it.
Replies
1
Boosts
0
Views
42
Activity
6d
Structuring Multi-Step Confirmations and Disambiguation in Siri Voice UX
We want to support voice workflows that require safety confirmations (e.g., 'Siri, prune my stale database caches' followed by Siri asking 'This will delete 42 entries, are you sure you want to proceed?'). How do we cleanly model multi-turn dialogs in Swift using AppIntent without forcing the user to re-trigger Siri? What is the correct way to throw dialog prompts or handle dynamic parameter disambiguation (e.g., if the user says 'Siri, check status' and multiple daemon instances are running, how do we return a list of choices to Siri to read out loud)?
Replies
1
Boosts
0
Views
37
Activity
6d
Use ShowInAppSearchResultsIntent with custom Parameters
I’m currently using ShowInAppSearchResultsIntent to open the app in a selectable view with the search results. The user can choose which view to pick via a @Parameter. Though with iOS 27 I can’t compile this intent anymore, because of this error: 'ShowInAppSearchResultsIntent' must only have a 'criteria' parameter What is the best practice to offer the same selection of the search view with iOS 27 and newer then? My Code import AppIntents @AppIntent(schema: .system.search) struct SearchAppIntent: ShowInAppSearchResultsIntent { static let searchScopes: [StringSearchScope] = [.general] var criteria: StringSearchCriteria // MARK: Parameters @Parameter(default: .loans) var target: SearchView? // MARK: Action Text static var parameterSummary: some ParameterSummary { Summary("Search \(\.$criteria) in \(\.$target)", table: "Shortcuts") } // MARK: Action @MainActor func perform() async throws -> some IntentResult { switch target { case .general, nil: // Open app search tab NavigationManager.shared.openAppSearch(with: criteria.term) case .contacts: // Open contacts tab NavigationManager.shared.openContactsSearch(with: criteria.term) } return .result() } }
Replies
2
Boosts
0
Views
62
Activity
6d
Handling Long-Running Background Actions and Network Sync in App Intents
Some of our intents involve initiating local-to-remote database synchronization (e.g., syncing local memory metrics to a remote MongoDB instance). This operation can take between 5 to 15 seconds depending on network conditions. What are the strict execution timeout limits for App Intents running in the background on macOS and iOS (especially when triggered via Siri without active UI)? If the operation exceeds the default limit, will Siri forcefully terminate the intent? How can we request background execution extension or hand off the work to a background download session/daemon while reporting progress back to Siri in real-time? Does the system support asynchronous yielding (Progress) so Siri can display a progress bar or status updates (e.g., 'Syncing 50%... Done') during execution?
Replies
1
Boosts
0
Views
51
Activity
6d
Intents for Live Activity Buttons/Toggles take a while to execute
Apple Music's Live Activity has instantly responsive prev/next/pause intents. Similar Buttons and Toggles on our Live Activity shimmer for a second or two as the LiveActivityIntent perform body executes light work in our main process, the content update is submitted and serialized, and that update finally renders. That makes our Live Activity a bit clunky. The delays also nixed some useful interactive feature ideas. The slow response is true even if our main process was running in the background with an active HKLiveWorkoutBuilder. Is this an expected limitation? Is there a strategy we should have taken to avoid this? (We do use optimistic animations via Toggle to soften the edges a bit.)
Replies
2
Boosts
0
Views
52
Activity
6d
"Answer" AppSchemeDomain available?
If you have an app that provides technical answers (similar to a LLM), is there AppSchemeDomain that's appropriate?
Replies
1
Boosts
0
Views
66
Activity
6d
Handling Raw User Input Prompts (Open-ended Dictation)
In some of our intents, we want the user to dictate a custom prompt or text body to be stored in the memory cache (e.g., 'Siri, add memory tag with content: Remember to invalid auth caches when schema bumps'). How can we define an AppIntent parameter that accepts open-ended dictation or arbitrary string blocks from Siri's speech-to-text translator? Are there length limits or language restrictions on string parameters parsed by Siri?
Replies
1
Boosts
0
Views
49
Activity
6d
Flexibility in adoption AppEntity Schemas
Hi, I have a list of contacts that I store in my app. Can I list all the contacts and their properties into the new Siri by adopting this schema? @available(iOS 27.0, *) @AppEntity(schema: .messages.messagePerson) struct CJPersonAppIndexedEntity: IndexedEntity { ... } It's not really part of the 'messages' domain, but is it correct to use? Also, if I have a concept of a 'task' and 'file' in my app? Can I use the .reminders domain and use .createreminder intent, and mix that 'reminders' and 'files' domains with the 'messages' domain, all in the same app?
Replies
1
Boosts
0
Views
43
Activity
6d
How to show correct title for intents in Spotlight
In my plant watering app, I created a PlantEntity and a WaterPlantIntent that edits the plant to mark it as watered. PlantEntity conforms to AppEntity and IndexedEntity. I also created OpenPlantIntent that conforms to OpenIntent. My AppShortcutsProvider is implemented like so: struct ShortcutsProvider: AppShortcutsProvider { static var appShortcuts: [AppShortcut] { return [ AppShortcut( intent: WaterPlantIntent(), phrases: [ "Water plant in \(.applicationName)", "Water \(\.$plantEntity) in \(.applicationName)" ], shortTitle: "Water Plant", systemImageName: "drop.fill" ) ] } } Now, when I search in Spotlight for a plant name, two results are shown but both are titled the name of the plant. The one in the App Name section just opens the plant in the app but the one in the Top Hit section actually performs the WaterPlantIntent! There's no indication that would occur. I expected it to be titled Water Plant Name so users know what shortcut is being run. Is there a way I can configure my code to show the proper title here?
Replies
3
Boosts
0
Views
73
Activity
6d
Will Siri AI only use AppIntents that have AppSchemas when "vibe coding" shortcuts?
I have an app that allows users to create charts and add series to them using AppIntents in Shortcuts. I couldn't find any AppSchemas that would match my entities in the official documentation. While "vibe coding" shortcuts, will Siri AI only access app intents through AppSchemas? If 1 is true, is there any alternative to AppSchemas to make AppIntents discoverable by Siri?
Replies
1
Boosts
0
Views
108
Activity
6d
.md summaries of WWDC for Claude
Are there any official .md summaries for Claude? How should we tell our Agents where the source of truth for WWDC new code is?
Replies
1
Boosts
0
Views
105
Activity
6d
synonyms API on DisplayRepresentation
For schema conforming App Entities, does the synonyms API on DisplayRepresentation feed into Siri AI's natural language understanding of your app's entities?
Replies
2
Boosts
0
Views
46
Activity
6d