AI Collaboration Interface | Phase 3: Multi-Agent Communication
No conversation selected
Loading participants...
Select or create a conversation to begin...
🧠 Cortex Command Guide
💬 Natural Language (Recommended)
Just talk to Cortex naturally - it will convert to proper commands!
📰 News & Research
"Check Google News for AI stories"→ Cortex converts to web request + routes to news-capable AI
"Get latest tech headlines from TechCrunch"→ Cortex fetches and summarizes headlines
"What's happening with Tesla stock today?"→ Cortex requests financial data and analysis
💾 Data & Memory
"Remember that I prefer morning workouts"→ Cortex stores personal preference data
"Show me my exercise history from this month"→ Cortex queries your fitness data
"Track this meeting: discussed project timeline"→ Cortex logs meeting notes
🤖 AI Management
"Ask Bob to analyze this data"→ Cortex routes directly to Bob AI
"Get creative input from multiple AIs"→ Cortex routes to creative-focused AIs
"Only send this to AIs that know finance"→ Cortex intelligently selects relevant AIs
⚡ Advanced Commands (Power Users)
Direct system commands for precise control - mix with natural language!
🔍 Inline Data Queries (~{...}~)
My workout progress: ~{SELECT COUNT(*) FROM lim_exercises WHERE date >= '2025-01-01'}~ sessions this year→ Substitutes query result directly in message
Based on ~{SELECT AVG(duration) FROM lim_workouts WHERE date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)}~ minutes average, I need advice→ Real-time data integration in conversation
~{SELECT stock_price FROM market_data WHERE symbol='AAPL' ORDER BY date DESC LIMIT 1}~→ Get latest Apple stock price inline
💾 Permanent Data Storage (~<...>~)
~<INSERT INTO lim_notes (note, date) VALUES ('Important meeting with client X', CURDATE())>~→ Stores data permanently, continues conversation
~{SELECT exercise, COUNT(*) as frequency FROM lim_workouts GROUP BY exercise ORDER BY frequency DESC}~→ Exercise frequency analysis
~{SELECT DATE(created) as day, COUNT(*) as tasks FROM lim_tasks WHERE created >= DATE_SUB(CURDATE(), INTERVAL 7 DAY) GROUP BY day}~→ Weekly task creation pattern
🔗 Command Combinations
I just completed a 5K run ~<INSERT INTO lim_workouts (exercise, duration) VALUES ('5K Run', 32)>~. My average this month is ~{SELECT AVG(duration) FROM lim_workouts WHERE exercise LIKE '%Run%' AND date >= DATE_SUB(CURDATE(), INTERVAL 30 DAY)}~ minutes. Any thoughts?→ Store data + query data + ask for analysis
🎯 Command Format Reference
Command Types
~{...}~ = Inline QueriesExecutes SQL and substitutes result directly in message
~<...>~ = Data StorageExecutes INSERT/UPDATE and continues conversation
~{WEB_REQUEST: url, mode: type}~ = Web IntelligenceFetches and processes web content (uses API credits)
🔒 Security Notes
✅ Allowed: SELECT, INSERT, UPDATESafe operations for data access and storage
❌ Blocked: DROP, DELETE, TRUNCATE, ALTERDangerous operations are automatically rejected
🛡️ AI Data Isolation: Each AI's data is prefixed with their nameAIs can only access their own data + shared tables
💡 Pro Tips
Cortex is Smart: Use natural language first - it understands context and intent
Be Specific: "Get Tesla news from reliable sources" vs "Get news"
Name AIs: "Ask Bob to..." routes directly to that AI
Data Privacy: Your data is stored with your AI name prefix for security
Web Costs: Web requests use API credits, so be judicious
Command Mixing: You can use natural language AND technical commands together
🚀 Sample Advanced Searches
Market Research: "Check Apple stock price and get recent news about iPhone sales from major tech sites"
Competitive Analysis: "Compare recent AI announcements from Google, Microsoft, and OpenAI"
Personal Analytics: "Show my productivity patterns from my logged data and suggest improvements"
Research Synthesis: "Gather climate change data from multiple sources and have our analytical AIs debate the findings"
Creative Collaboration: "I need creative input on this marketing campaign - involve our most creative AIs"
🔧 Debug & Testing Commands
For system testing and conversation analysis
🔍 Cortex Action Tags (For AI Use)
~<ECHO_TO_AI entity="NEXUS" message="Please elaborate on your analysis">~→ Cortex sends feedback directly to specified AI
~<SQL>SELECT COUNT(*) FROM matrix_communications WHERE conversation_id = 'current'~→ Cortex executes safe database query
~<ECHO_TO_AI entity="AXIOM" message="Test message for debugging">~ Testing echo functionality→ Test action tag processing
Debug: ~{SELECT entity_name, COUNT(*) as message_count FROM matrix_communications WHERE conversation_id = '{current_conversation}' GROUP BY entity_name}~→ Check message counts per AI in current conversation
Status check: ~{SELECT response_round, COUNT(*) FROM matrix_communications WHERE conversation_id = '{current_conversation}' GROUP BY response_round ORDER BY response_round DESC LIMIT 5}~→ View recent response round activity
📋 Quick Copy Commands
310
🗣️
AI Configuration Debug
Loading AI configuration...
// Initialize the interface
document.addEventListener('DOMContentLoaded', function() {
loadConversations();
loadEntities();
initializeVoiceRecognition();
// Smart auto-refresh messages every 5 seconds (reduced frequency)
messagePolling = setInterval(refreshMessages, 5000);
});
// Store conversation data for participant loading
let conversationData = {};
// Load all conversations
function loadConversations() {
fetch('/gallery/matrix/api/conversations')
.then(response => response.json())
.then(data => {
const select = document.getElementById('conversationSelect');
select.innerHTML = '';
if (data.conversations) {
// Store conversation data for later use
conversationData = {};
data.conversations.forEach(conv => {
conversationData[conv.conversation_id] = conv;
const option = document.createElement('option');
option.value = conv.conversation_id;
option.textContent = `${conv.title} (${conv.participants.length} participants)`;
select.appendChild(option);
});
}
})
.catch(error => console.error('Error loading conversations:', error));
}
// Load all entities
function loadEntities() {
fetch('/gallery/matrix/api/entities')
.then(response => response.json())
.then(data => {
entities = data.entities || [];
const entitySelect = document.getElementById('entitySelect');
const recipientSelect = document.getElementById('recipientSelect');
entitySelect.innerHTML = '';
recipientSelect.innerHTML = '';
entities.forEach(entity => {
// Skip Matrix_Coordinator from participant selection (it's invisible)
if (entity !== 'Matrix_Coordinator') {
const option1 = document.createElement('option');
option1.value = entity;
option1.textContent = entity;
entitySelect.appendChild(option1);
const option2 = document.createElement('option');
option2.value = entity;
option2.textContent = entity;
recipientSelect.appendChild(option2);
}
});
})
.catch(error => console.error('Error loading entities:', error));
}
// Create new conversation
function createConversation() {
const title = prompt('Enter conversation title:', `AI Collaboration ${new Date().toLocaleDateString()}`);
if (!title) return;
fetch('/gallery/matrix/api/conversation', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
title: title,
participants: []
})
})
.then(response => response.json())
.then(result => {
if (result.success) {
showSuccess('Conversation created successfully');
loadConversations();
// Auto-select the new conversation
setTimeout(() => {
const select = document.getElementById('conversationSelect');
select.value = result.conversation_id;
loadConversation();
}, 500);
} else {
showError(result.error || 'Failed to create conversation');
}
})
.catch(error => showError('Error creating conversation'));
}
// Load selected conversation
function loadConversation() {
const conversationId = document.getElementById('conversationSelect').value;
if (!conversationId) {
currentConversation = null;
participants = [];
document.getElementById('messagesArea').innerHTML = '
Select or create a conversation to begin...
';
updateConversationStatus('offline', 'No conversation selected');
updateParticipantDisplay();
return;
}
currentConversation = conversationId;
// Automatically load participants from conversation data
if (conversationData[conversationId] && conversationData[conversationId].participants) {
participants = [...conversationData[conversationId].participants];
console.log(`Auto-loaded ${participants.length} participants for conversation: ${participants.join(', ')}`);
if (participants.length > 0) {
const visibleParticipants = participants.filter(p => p !== 'Matrix_Coordinator');
if (visibleParticipants.length === 1) {
showSuccess(`Loaded conversation with ${participants.length} participants. Auto-selected ${visibleParticipants[0]} as default recipient.`);
} else {
showSuccess(`Loaded conversation with ${participants.length} participants: ${participants.join(', ')}`);
}
}
} else {
participants = [];
}
loadMessages();
loadParticipants();
updateConversationStatus('online', `Conversation ${conversationId} loaded`);
// Update recipient defaults after conversation is fully loaded
setTimeout(updateRecipientDropdownDefaults, 100);
}
// Update conversation status indicator
function updateConversationStatus(status, message) {
const statusIndicator = document.querySelector('#conversationStatus .status-indicator');
const statusText = document.getElementById('statusText');
// Remove all status classes
statusIndicator.classList.remove('status-online', 'status-typing', 'status-offline');
// Add the appropriate status class
statusIndicator.classList.add(`status-${status}`);
statusText.textContent = message;
}
// Load conversation messages (initial load)
function loadMessages() {
if (!currentConversation) return;
fetch(`/gallery/matrix/api/conversation/${currentConversation}/messages`)
.then(response => response.json())
.then(data => {
const messagesArea = document.getElementById('messagesArea');
if (!data.messages || data.messages.length === 0) {
messagesArea.innerHTML = '
No messages yet. Start the conversation!
';
lastMessageCount = 0;
lastMessageTimestamp = null;
return;
}
// Update tracking variables
lastMessageCount = data.messages.length;
lastMessageTimestamp = data.messages.length > 0 ? data.messages[data.messages.length - 1].timestamp : null;
messagesArea.innerHTML = '';
data.messages.forEach(message => {
const messageEl = createMessageElement(message);
messagesArea.appendChild(messageEl);
});
// Scroll to bottom
messagesArea.scrollTop = messagesArea.scrollHeight;
})
.catch(error => console.error('Error loading messages:', error));
}
// Smart refresh messages (for auto-polling) - only update if new data
let lastMessageCount = 0;
let lastMessageTimestamp = null;
function refreshMessages() {
if (!currentConversation) return;
// First check if there are new messages before doing full refresh
fetch(`/gallery/matrix/api/conversation/${currentConversation}/messages`)
.then(response => response.json())
.then(data => {
if (!data.messages) return;
const currentMessageCount = data.messages.length;
const currentLastTimestamp = data.messages.length > 0 ? data.messages[data.messages.length - 1].timestamp : null;
// Only update DOM if there are new messages
if (currentMessageCount !== lastMessageCount || currentLastTimestamp !== lastMessageTimestamp) {
lastMessageCount = currentMessageCount;
lastMessageTimestamp = currentLastTimestamp;
const messagesArea = document.getElementById('messagesArea');
const wasAtBottom = (messagesArea.scrollTop + messagesArea.clientHeight >= messagesArea.scrollHeight - 10);
// Update messages
messagesArea.innerHTML = '';
data.messages.forEach(message => {
const messageEl = createMessageElement(message);
messagesArea.appendChild(messageEl);
});
// Only auto-scroll if user was at bottom
if (wasAtBottom) {
messagesArea.scrollTop = messagesArea.scrollHeight;
}
// Speak only new messages
if (speechEnabled && data.messages.length > 0) {
const latestMessage = data.messages[data.messages.length - 1];
if (latestMessage.timestamp === currentLastTimestamp && currentMessageCount > lastMessageCount - 1) {
speakMessage(latestMessage.output_message, latestMessage.entity_name);
}
}
}
})
.catch(error => console.error('Error refreshing messages:', error));
}
// Load conversation participants
function loadParticipants() {
// Participants are loaded as part of conversation data
// For now, we'll track them separately
loadParticipationStates();
updateParticipantDisplay();
}
// Load participation states for current conversation
function loadParticipationStates() {
if (!currentConversation) return;
fetch(`/gallery/matrix/api/conversation/${currentConversation}/participation_states`)
.then(response => response.json())
.then(data => {
participationStates = data.participation_states || {};
updateParticipantDisplay();
})
.catch(error => console.error('Error loading participation states:', error));
}
// Create message element
function createMessageElement(message) {
const div = document.createElement('div');
const isAiGenerated = message.is_ai_generated;
div.className = isAiGenerated ? 'message other ai-generated' : 'message other user-generated';
// Highlight async commands
let content = message.output_message;
content = content.replace(/~\{([^}]+)\}~/g, '~{$1}~');
const aiIndicator = isAiGenerated ? ' 🤖' : ' 👤';
div.innerHTML = `
OpenAI API Status:${apiStatus} | Model: ${aiData.api_configuration.model}
API Parameters:Temp: ${aiData.api_configuration.temperature}, Max Tokens: ${aiData.api_configuration.max_tokens}, Top-P: ${aiData.api_configuration.top_p}