TeaPoly’s gists

TeaPoly / deepseek_v3_moe.py

Last active April 8, 2025 10:30

DeepSeek V3 MoE with aux-loss-free and sequence aux loss.

	#!/usr/bin/python
	# -- coding: utf-8 --
	# Copyright 2024 Lucky Wong
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#

TeaPoly / absmean_binarize.py

Last active June 14, 2024 07:43

	#!/usr/bin/python
	# -- coding: utf-8 --

	import torch


	def absmean_binarize(x, contract_dims, centralize=False, eps=1e-8):
	if centralize:
	mean = torch.mean(x, dim=contract_dims, keepdim=True)
	x = x - mean

TeaPoly / i2wasymsc.py

Last active June 14, 2024 07:44

	#!/usr/bin/python
	# -- coding: utf-8 --

	import torch


	def round_ste(x):
	# STE for gradient
	return torch.floor(x + 0.5).detach() + (x - x.detach())

TeaPoly / attention_alignment.py

Created May 30, 2023 02:53

Deep model with built-in self-attention alignment for acoustic echo cancellation

	#!/usr/bin/python
	# -- coding: utf-8 --
	# Copyright 2023 Lucky Wong
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#

TeaPoly / plot_audio.py

Last active April 25, 2023 08:39

plot audio in SummaryWriter

	#!/usr/bin/python
	# -- coding: utf-8 --
	# Copyright 2022 Lucky Wong
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#

TeaPoly / lfr.py

Created March 29, 2023 03:11

apply the stacking to inputs

	#!/usr/bin/python
	# -- coding: utf-8 --
	# Copyright 2022 Lucky Wong
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#

TeaPoly / neon_to_rvv.h

Created February 20, 2023 08:25

Neon to RISC-V V

	/*
	* Copyright (c), Lucky Wong.
	*
	* MIT License (MIT), http://opensource.org/licenses/MIT
	* Full license can be found in the LICENSE file
	*/

	#include <riscv_vector.h>

	typedef vfloat32m4_t float32x4_t;

TeaPoly / get_wav_duration.py

Last active December 9, 2022 07:36

Fast way to get duration for WAVE PCM format audio.

	#!/usr/bin/env python3
	# Copyright 2022 Lucky Wong
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#
	# Unless required by applicable law or agreed to in writing, software

TeaPoly / mwer_loss.py

Last active November 7, 2022 06:07

The implementation of Minimum Word Error Rate Training loss (MWER) based on negative sampling strategy from <Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition>

	#!/usr/bin/python
	# -- coding: utf-8 --
	# Copyright 2022 Lucky Wong
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#

TeaPoly / multi_channel_attention.py

Last active September 14, 2022 07:12

The implementation of self-attention which is helpful to improve multi-channel KWS performance as well as reduce computational complexity. Inspired from paper Joint Ego-Noise Suppression and Keyword Spotting on Sweeping Robots.

	#!/usr/bin/env python3
	# Copyright 2022 Lucky Wong
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#
	# Unless required by applicable law or agreed to in writing, software

Lucky Wong TeaPoly