Add documentation for external sensitive media detection API

- Create API documentation explaining request/response format
- Add JSDoc comments to AiService methods
- Provide example implementation using nsfwjs

Co-authored-by: syuilo <4439005+syuilo@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot] 2025-11-18 07:14:21 +00:00
parent d5963562cb
commit dce6f46334
2 changed files with 137 additions and 0 deletions

View File

@ -0,0 +1,117 @@
# Sensitive Media Detection External Service API
This document describes the API contract for an external sensitive media detection service that can be used with Misskey.
## Endpoint
The external service should expose a single HTTP POST endpoint at the URL configured in the Misskey admin settings (`sensitiveMediaDetectionProxyUrl`).
## Request
### Method
`POST`
### Headers
- `Content-Type: application/json`
### Body
JSON object with the following structure:
```json
{
"image": "base64-encoded-image-data"
}
```
Where `image` is the base64-encoded binary data of the image file to analyze.
## Response
### Success (HTTP 200)
JSON object with the following structure:
```json
{
"predictions": [
{
"className": "Neutral",
"probability": 0.95
},
{
"className": "Drawing",
"probability": 0.03
},
{
"className": "Sexy",
"probability": 0.01
},
{
"className": "Porn",
"probability": 0.005
},
{
"className": "Hentai",
"probability": 0.005
}
]
}
```
The `predictions` array should contain objects with:
- `className`: One of `"Neutral"`, `"Drawing"`, `"Sexy"`, `"Porn"`, or `"Hentai"`
- `probability`: A number between 0 and 1 indicating the confidence level
The classification names and behavior should match the [nsfwjs](https://github.com/infinitered/nsfwjs) library.
### Error
On error, the external service can return any HTTP error status code. Misskey will treat the request as failed and will not mark the file as sensitive (allowing the upload to proceed).
## Timeout
Requests will timeout after 10 seconds. The external service should respond within this time limit.
## Example Implementation
A simple example using nsfwjs:
```javascript
const express = require('express');
const nsfw = require('nsfwjs');
const tf = require('@tensorflow/tfjs-node');
const app = express();
app.use(express.json({ limit: '50mb' }));
let model;
async function loadModel() {
model = await nsfw.load();
}
app.post('/', async (req, res) => {
try {
const { image } = req.body;
const buffer = Buffer.from(image, 'base64');
const tfImage = await tf.node.decodeImage(buffer, 3);
const predictions = await model.classify(tfImage);
tfImage.dispose();
res.json({ predictions });
} catch (error) {
res.status(500).json({ error: error.message });
}
});
loadModel().then(() => {
app.listen(3000, () => {
console.log('Sensitive media detection service running on port 3000');
});
});
```
## Benefits of External Service
- **Reduced Memory Usage**: nsfwjs and TensorFlow can be memory-intensive. Running them separately reduces Misskey's memory footprint.
- **Scalability**: The detection service can be scaled independently.
- **Alternative Implementations**: Operators can use different detection models or services (e.g., cloud-based APIs, more efficient models).
- **Isolation**: Issues with the detection service don't affect Misskey's core functionality.

View File

@ -35,6 +35,16 @@ export class AiService {
) {
}
/**
* Detect sensitive content in an image.
*
* If an external proxy URL is configured (sensitiveMediaDetectionProxyUrl),
* this method will use the external service. Otherwise, it will use the
* built-in nsfwjs model.
*
* @param source - Path to the image file or a Buffer containing the image data
* @returns Array of predictions or null if detection fails
*/
@bindThis
public async detectSensitive(source: string | Buffer): Promise<nsfw.PredictionType[] | null> {
// If external service is configured, use it
@ -78,6 +88,16 @@ export class AiService {
}
}
/**
* Detect sensitive content using an external proxy service.
*
* Sends the image as base64-encoded data to the external service
* and expects a response in the same format as nsfwjs.
*
* @param source - Path to the image file or a Buffer containing the image data
* @returns Array of predictions or null if the request fails
* @see docs/sensitive-media-detection-api.md for API contract details
*/
@bindThis
private async detectSensitiveWithProxy(source: string | Buffer): Promise<nsfw.PredictionType[] | null> {
try {