반응형
MediaPipe pose는 ML Kit 포즈 감지 API를 지원하는 BlazePose연구를 활용하여 RGB 비디오 프레임에서 전신에 대한 33개의 3D 랜드마크와 배경 분할 마스크를 추론하고 신뢰도가 높은 신체 포즈 추적을 위한 ML 솔루션이다.
BlazePose 논문을 나름 해석해보았으니 참고 하실분은 링크를 클릭해주길 바란다!
https://machineindeep.tistory.com/34
BlazePose: On-device Real-time Body Pose tracking 리뷰
pose estimation 프로젝트를 진행하다 mediapipe라는 프레임워크를 알게되었다. 나같은 경우에는 여러 사람을 multi tracking 하는게 아닌 한 사람만 감지하는 single tracking 이 필요했고 mediapipe blazepose는..
machineindeep.tistory.com
google mediapipe 공식 홈페이지 (https://google.github.io/mediapipe/solutions/pose.html)
Pose
Cross-platform, customizable ML solutions for live and streaming media.
google.github.io
이 주소의 코드를 그대로 사용하셔도 실행이 되시는 분들은 상관없지만
저는 실행 시 error가 발생하기 때문에 Stack Overflow의 도움을 받아 코드를 재작성 하였다.
아래 코드를 html문서에 붙여넣기 하고 캠코더를 사용하여 누구든지 쉽게 사용할 수 있다.
<!DOCTYPE html>
<html>
<head>
<style>
body {
bottom: 0;
font-family: 'Titillium Web', sans-serif;
color: white;
left: 0;
margin: 0;
position: absolute;
right: 0;
top: 0;
transform-origin: 0px 0px;
overflow: hidden;
}
.container {
position: absolute;
background-color: #596e73;
width: 100%;
max-height: 100%;
}
.input_video {
display: none;
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
&.selfie {
transform: scale(-1, 1);
}
}
.output_canvas {
max-width: 100%;
display: block;
position: relative;
left: 0;
top: 0;
}
</style>
<meta charset="utf-8">
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils@0.3.1632432234/camera_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils@0.6.1629159505/control_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils_3d@0.3.1635987963/control_utils_3d.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils@0.3.1620248257/drawing_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/pose@0.5.1635988162/pose.js" crossorigin="anonymous"></script>
<!-- <script src="https://cdn.jsdelivr.net/npm/@mediapipe/hands@0.3.1632795355/hands.js" crossorigin="anonymous"></script>-->
</head>
<body>
<div class="container">
<video class="input_video"></video>
<canvas class="output_canvas" width="1280px" height="720px"></canvas>
<div class="landmark-grid-container"></div>
</div>
</body>
</html>
<script type="module">
const videoElement = document.getElementsByClassName('input_video')[0];
const canvasElement = document.getElementsByClassName('output_canvas')[0];
const canvasCtx = canvasElement.getContext('2d');
const landmarkContainer = document.getElementsByClassName('landmark-grid-container')[0];
const grid = new LandmarkGrid(landmarkContainer);
let posenet_keypoints = [0, 2, 5, 7, 8, 11, 12, 13, 14, 15, 16, 23, 24, 25, 26, 27, 28];
function onResults(results) {
if (!results.poseLandmarks) {
grid.updateLandmarks([]);
return;
}
canvasCtx.save();
canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
/*
canvasCtx.drawImage(results.segmentationMask, 0, 0,
canvasElement.width, canvasElement.height);
*/
// Only overwrite existing pixels.
canvasCtx.globalCompositeOperation = 'source-in';
canvasCtx.fillStyle = '#00FF00';
canvasCtx.fillRect(0, 0, canvasElement.width, canvasElement.height);
// Only overwrite missing pixels.
canvasCtx.globalCompositeOperation = 'destination-atop';
canvasCtx.drawImage(
results.image, 0, 0, canvasElement.width, canvasElement.height);
console.log(results.poseLandmarks.slice(posenet_keypoints));
canvasCtx.globalCompositeOperation = 'source-over';
drawConnectors(canvasCtx, results.poseLandmarks, POSE_CONNECTIONS,
{color: '#00FF00', lineWidth: 4});
drawLandmarks(canvasCtx, results.poseLandmarks,
{color: '#FF0000', lineWidth: 2});
canvasCtx.restore();
grid.updateLandmarks(results.poseWorldLandmarks);
}
const pose = new Pose({locateFile: (file) => {
return `https://cdn.jsdelivr.net/npm/@mediapipe/pose@0.5.1635988162/${file}`;
}});
pose.setOptions({
modelComplexity: 1,
smoothLandmarks: true,
enableSegmentation: false,
smoothSegmentation: false,
minDetectionConfidence: 0.5,
minTrackingConfidence: 0.5
});
pose.onResults(onResults);
const camera = new Camera(videoElement, {
onFrame: async () => {
await pose.send({image: videoElement});
},
width: 1280,
height: 720
});
camera.start();
</script>반응형
'ML | DL > Pose estimation' 카테고리의 다른 글
| PoseNet keypoint로 skeleton 그리기 (1) | 2022.03.17 |
|---|---|
| PoseNet keypoint로 신체 각도 구하기 (4) | 2022.03.17 |
| ml5 PoseNet KNN Classification 사용방법 (0) | 2022.03.17 |
| Pose Similarity - Weight Distance Python 코드 (0) | 2022.03.17 |
댓글