[WIP] Update `LoraConfig` for KaSA implementation #2698

iambogeumkim · 2025-08-02T05:41:15Z

I was delayed in updating the code because I was focusing on company work, but now I'm planning to resume the project in earnest. If I have any questions about implementing the code, may I continue to ask you?

I apologize for opening a new pull request, as the previous one was closed 🥲 Thank you for your understanding.

BenjaminBossan

Thank you for resuming your work on KaSA.

Implementation-wise, we need to take a different approach. Right now, KaSA is just added to the normal LoRA code, but we only want to activate it if the user opts in. Therefore, it should be implemented in a separate class, something like KasaVariant, in peft/tuners/lora/variants.py. Please check how DoRA is implemented and use a similar approach, as I have detailed in my previous comment. If anything is unclear, feel free to ask.

github-actions · 2025-09-01T15:04:02Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

BenjaminBossan · 2025-09-01T15:12:53Z

gentle ping @nsbg

… variants.py

iambogeumkim · 2025-09-02T12:14:03Z

Thank you for your alert!

I spent some time looking over the KaSA paper and code to get ready for more serious work, but it does seem pretty difficult 🥲 My goal is to upload code that's ready for review before the end of September, so I'm going to try even harder.

Right now, I'm stuck at the 'Extend LoRA variant resolution' stage you mentioned. Honestly, this seems like the most important part, but it's hard for me to figure out where to start—specifically, which file and class I should work on first. Could you help me with this?

BenjaminBossan · 2025-09-02T14:04:09Z

That's great to see, thanks for picking this back up.

Right now, I'm stuck at the 'Extend LoRA variant resolution' stage you mentioned. Honestly, this seems like the most important part, but it's hard for me to figure out where to start—specifically, which file and class I should work on first. Could you help me with this?

You're already on the right track, you added KasaLinearVariant, which is the most important step. There are definitely some changes required there, as there is some code that is only relevant for DoRA and can be removed for KaSA. But we can leave that as is for now.

Next about resolving the variants. As a first step, let's revert the changes you made to lora/layer.py and start fresh. We don't need a self.use_kasa attribute, we only have self.use_dora for backwards compatibility, as we didn't have LoRA variants when we first implemented DoRA.

Then let's look at these lines in lora.Linear:

peft/src/peft/tuners/lora/layer.py

Lines 636 to 642 in a3197b1

    
           def resolve_lora_variant(self, *, use_dora: bool, **kwargs) -> Optional[LoraVariant]: 
        
               if not use_dora: 
        
                   return None 
        
               from .variants import DoraLinearVariant 
        
               return DoraLinearVariant()

Here we need to extend the functionality to add KaSA. The updated method could be something like:

    def resolve_lora_variant(self, *, use_dora: bool, use_kasa: bool, **kwargs) -> Optional[LoraVariant]:
        if use_dora and use_kasa:
            raise ValueError("Cannot use DoRA and KaSA at the same time, please choose only one.")

        variant = None
        if use_dora:
            from .variants import DoraLinearVariant

            variant = DoraLinearVariant()
        elif use_kasa:
            ...

        return variant

Does that make sense? Similarly, we'd have to update the resolve_lora_variant methods of other LoRA layers, depending on whether they work with KaSA or not (I'm not sure if KaSA works with Conv2d etc.).

I would suggest that you work on this as a next step, then we'll see what else needs to be done.

iambogeumkim · 2025-09-04T14:57:54Z

wow I really appreciate your sincere feedback. I'll read your advice carefully and then move forward 🤗

iambogeumkim · 2025-09-08T18:39:23Z

@BenjaminBossan I modified the code in the files below based on what you explained. Please give me feedback if there are parts that still need fixing, and then we can discuss the next steps.

1. variants.py

Completed updates to methods in the KasaLinearVariants class

2. layer.py

In the LoraLayer class, added self.use_kasa[adapter_name] = use_kasa inside the update_layer method
In the Linear class, added KaSA handling logic inside the get_delta_weight method

BenjaminBossan

Thanks for integrating my feedback. I gave this another review and noted the next few changes that are necessary. Please check my comments.

Apart from this, the branch is now encountering merge conflicts. Could you please bring your fork up-to-date with the remote and then merge with, or rebase on, the latest main branch from PEFT? If you have questions on how to resolve the merge conflicts, don't hesitate to ask.

Furthermore, please always run make style on your changes before pushing to make our linter happy.

More of a note for myself: Since KaSA updates the base weights of the model, we will have to take extra care to ensure that it works correctly when saving and loading the adapter.

BenjaminBossan · 2025-09-09T13:44:26Z

src/peft/tuners/lora/layer.py


        """
-        return None
+        if use_dora and use_kasa:


Let's undo the changes in this method body and return None. Instead, since this KaSA layer is implemented for Linear only, add the logic to lora.Linear.resolve_lora_variant instead.

Also, we should update the resolve_lora_variant methods of the other layer types like lora.Embedding.resolve_lora_variant to accept the use_kasa argument but raise an error if it's True. Otherwise, users may add it to non-supported layers and not notice that it doesn't actually do anything there.

BenjaminBossan · 2025-09-09T13:45:08Z

src/peft/tuners/lora/layer.py

+        ############ kasa ############# 
+        self.lora_diag[adapter_name] = nn.Parameter(torch.randn(r), requires_grad=True)
+
+        weight = self.get_base_layer().weight
+        dtype = weight.dtype
+        svd_rank = self.in_features - r
+        weight = weight.to(torch.float32)
+        U, S, Vh = torch.linalg.svd(weight.data, full_matrices=False)
+        U_principle, S_principle, Vh_principle = U[:, :svd_rank], S[:svd_rank], Vh[:svd_rank, :]
+        self.get_base_layer().weight.data = (U_principle @ torch.diag(S_principle) @ Vh_principle).to(dtype)

+        #########################


All of this can be removed, since it's part of KasaLinearVariant.init, right?

BenjaminBossan · 2025-09-09T13:47:30Z

src/peft/tuners/lora/variants.py

+        # initialize lora_diag
+        module.lora_diag[adapter_name] = nn.Parameter(torch.randn(module.r[adapter_name]), requires_grad=True)
+
+        # SVD


Let's add a reference here, so that we know the origin:
# see https://github.com/juyongjiang/KaSA/blob/f85e88c22d0fa4cb8ab2923d7c2bf1bbec152da3/peft/src/peft/tuners/lora/layer.py#L132

# initialize lora_diag module.lora_diag[adapter_name] = nn.Parameter(torch.randn(module.r[adapter_name]), requires_grad=True) # see https://github.com/juyongjiang/KaSA/blob/f85e88c22d0fa4cb8ab2923d7c2bf1bbec152da3/peft/src/peft/tuners/lora/layer.py#L132 # SVD

I put it in here, how is it?

BenjaminBossan · 2025-09-09T13:51:30Z

src/peft/tuners/lora/variants.py

+    @staticmethod
+    def merge_safe(module: Linear, active_adapter: str, orig_weight: torch.Tensor) -> torch.Tensor:
+        delta_weight = module.get_delta_weight(active_adapter)
+        return orig_weight + delta_weight
+
+    @staticmethod
+    def merge_unsafe(module: Linear, active_adapter: str, orig_weight: torch.Tensor) -> None:
+        delta_weight = module.get_delta_weight(active_adapter)
+        orig_weight.data += delta_weight
+
+    @staticmethod
+    def unmerge(module: Linear, active_adapter: str, orig_weight: torch.Tensor) -> torch.Tensor:
+        delta_weight = module.get_delta_weight(active_adapter)
+        return orig_weight - delta_weight


KaSA should have an influence on the merged weights, should it not?

Although this PR is closed, it seems I've incorporated everything else except for this comment (of course, you'd have to look at the code). Could you explain this question in more detail?

BenjaminBossan · 2025-09-09T13:53:07Z

src/peft/tuners/lora/variants.py

+            x = dropout(x)
+
+        # KaSA calculation
+        lora_output = lora_B(torch.einsum('ijk,kl->ijl', lora_A(x), diag)) * scaling


Again, let's add a reference:
# see https://github.com/juyongjiang/KaSA/blob/f85e88c22d0fa4cb8ab2923d7c2bf1bbec152da3/peft/src/peft/tuners/lora/layer.py#L602C21-L602C110

# KaSA calculation # see https://github.com/juyongjiang/KaSA/blob/f85e88c22d0fa4cb8ab2923d7c2bf1bbec152da3/peft/src/peft/tuners/lora/layer.py#L602C21-L602C110 lora_output = lora_B(torch.einsum('ijk,kl->ijl', lora_A(x), diag)) * scaling return result + lora_output

I inserted this near where the actual calculation logic begins, rather than just in an empty space. I think this is a bit better.

iambogeumkim · 2025-09-16T17:02:08Z

@BenjaminBossan oh I didn't mean to close the branch, but it seems to have closed while I was merging with the main branch. I guess I'll have to open a new PR, right? 😰

+) when I tried to sync with the main branch, I ended up discarding all my commits, so did that cause it to close?

BenjaminBossan · 2025-09-17T08:55:18Z

oh I didn't mean to close the branch, but it seems to have closed while I was merging with the main branch. I guess I'll have to open a new PR, right? 😰

+) when I tried to sync with the main branch, I ended up discarding all my commits, so did that cause it to close?

I don't know what happened, but I could re-open the PR and there are some changes visible. Can you double check that everything looks as expected? If for some reason it's not what it's expected, you can create a new PR and push your local branch.

iambogeumkim · 2025-09-17T09:08:15Z

I usually handle merges in the terminal, and I suspect the pull request was closed because I accidentally wiped the commit history while using the 'Sync fork' feature on GitHub. I'll be more careful in the future. Thanks for reopening it.

I'll review the changes and open a new PR if needed. Sorry to keep bothering you with this.

BenjaminBossan · 2025-09-18T09:35:02Z

I'll review the changes and open a new PR if needed. Sorry to keep bothering you with this.

No worries. If the diff on this PR looks good, let me know and I'll do a review. Only open a new PR if for some reason, the code here does not correspond to what it should be.

BenjaminBossan

Thanks for the latest updates. There were still a few minor issues, please check. Also some further changes that are needed

- the resolve_lora_variant from lora.Linear has yet to check if use_kasa and return the KasaLinearVariant() in that case
- pass use_kasa here:
  
  peft/src/peft/tuners/lora/model.py
  
  Line 199 in f8aca0a
  
  "use_qalora": lora_config.use_qalora,
same here:

peft/src/peft/tuners/lora/model.py

Line 236 in f8aca0a

use_dora=lora_config.use_dora,

Note that you can run the tests locally to verify that they pass:

pytest tests/test_custom_models.py -k kasa -v

Once you're done with your changes, don't forget to call make style.

tests/test_custom_models.py

src/peft/tuners/lora/layer.py

src/peft/tuners/lora/variants.py

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

iambogeumkim · 2025-10-11T16:30:39Z

I have a question after local testing. I'll briefly share the test results and process.

1. 10 failures occurred

8 dtype mismatch issues, 2 AssertionError

2. dtype issues (all resolved ✅)

When testing cases where the model's data type changes, like in the test_forward_bfloat16 method, dtype mismatch issues occurred.
This was resolved by adding a part to match the dtype in a method within the KasaLinearVariants class.

3. AssertionError issues

An assert False error occurred in the test_disable_adapters method.
So, I added the following conditional statement to the forward method so that outputs_before == outputs_disabled holds True in the test_disable_adapters, by skipping KaSA calculation and returning only the base model output when adapters are disabled.
```
# Check if adapters are disabled
if module.disable_adapters:
  return result
```
However, assert False is still occurring.

But now that I think about it, looking at the current KasaLinearVariants class, it applies SVD to the weights and calculates from the init method. So I'm wondering if assert False is actually correct, or if I just tried to solve it in the wrong way.

BenjaminBossan

Thanks for your recent updates. Your observation is correct, due to KaSA modifying the base weights, it cannot be simply deactivated. To accommodate the tests, I would suggest to write a similar function to this one:

peft/tests/test_custom_models.py

Lines 1146 to 1148 in 2410f45

    
           def _skip_tests_with_multiple_adapters_with_target_parameters(config_cls, config_kwargs): 
        
               if (config_cls == LoraConfig) and config_kwargs.get("target_parameters"): 
        
                   pytest.skip("LoRA with multiple adapters with target_parameters is not supported")

Then, for those tests that don't work with KaSA, let's invoke this skipping logic.

We also have to document this KaSA property. Furthermore, we should consider warning the user if they try to deactivate KaSA, but I'm not sure yet if there is a good place to do that.

Moreover, with the changed base weights, it also means that we cannot use a KaSA adapter together with other adapters (e.g. with a normal LoRA adapter), right? There should be a check for this, which can be implemented in LoraModel:

    def _check_new_adapter_config(self, config: VeraConfig) -> None:
        super()._check_new_adapter_config(config)
        # add a check here that we cannot have multiple adapters if one of them uses KaSA

We then need a new test in test_initialization.py to test this new check.

However, I think it would be possible to have multiple adapters if they all use KaSA, right? The modification to the base weight would be identical for all KaSA adapters, so they should be able to coexist. The issue is that, right now, each new KaSA adapter would re-apply SVD and modify the base weight, which is not good. We would need to have a mechanism to detect that this has already happened and avoid applying it twice. WDYT?

iambogeumkim · 2025-11-04T15:35:22Z

Sorry for the delayed update; I've been working on this in bits and pieces due to personal scheduling. Since some time has passed, I'll summarize the feedback you previously gave me and my related work:

1. `_skip_test_disable_adapters` function

Added skip logic for test functions that do not work with KaSA adapters. (you can check in tests/test_custom_models.py)

2. `TestKasaInitialization` class

test_kasa_mixed_adapter_error method: When existing adapters are not KaSA, adding a new KaSA adapter should raise a ValueError.
test_kasa_mixed_adapter_error_reverse method: When existing adapters are KaSA, adding a non-KaSA adapter should raise a ValueError.

Both of these test functions passed because they successfully raised a ValueError. My understanding is that "PASSED" here means the ValueError was raised, which is the intended behavior for these functions. Please correct me if my understanding is wrong.

3. Logic to avoid repetitive SVD processes (Just an idea 💡)

I agree with your opinion that repeatedly modifying the base weights for the same KaSA adapter is not ideal. My current idea is to apply a cache, as shown below, and check if SVD has already been applied by checking the existence of this attribute. What do you think of this approach?

if not hasattr(module, "_kasa_svd_cache"):
    # First KaSA adapter: perform SVD and cache the result
    weight = module.get_base_layer().weight
    dtype = weight.dtype
    weight = weight.to(torch.float32)
    U, S, Vh = torch.linalg.svd(weight.data, full_matrices=False)
    module._kasa_svd_cache = (U, S, Vh)
else:
    # Reuse cached SVD result
    U, S, Vh = module._kasa_svd_cache
    dtype = module.get_base_layer().weight.dtype

BenjaminBossan

Thanks for the updates, we're moving towards the goal. I commented on some of the changes, please check.

My current idea is to apply a cache, as shown below, and check if SVD has already been applied by checking the existence of this attribute. What do you think of this approach?

I'm not sure it's needed, but I could be wrong. AFAICT, the SVD is only needed to update the base weight. If we add a second KaSA adapter, the base weight is already updated, so there is no need to cache U, S, Vh, we can just skip completely. Is my understanding correct?

Also, there are some merge conflicts on this PR now. I think they should be easy to resolve, but don't hesitate to ask if you have questions. Finally, before committing, don't forget to call make style.

src/peft/tuners/lora/model.py

tests/test_initialization.py

github-actions · 2025-11-29T15:03:59Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

iambogeumkim · 2025-11-29T15:05:16Z

Check

…apter types, enhancing compatibility checks in the initialization process.

…ve readability in LoraModel class.

…re SVD is applied only once, while also cleaning up whitespace in multiple locations.

iambogeumkim · 2025-12-06T08:07:36Z

@BenjaminBossan

I've addressed the points you mentioned, applied make style, and resolved the conflicts. Let me know if anything else needs to be updated.

Regarding the SVD value caching, I gave it some thought and realized I was stuck on the idea that 'caching is always efficient.' Since the base weights are already updated in the first adapter even when using multiple KaSA adapters, I realized we can simply reuse those values subsequently. So, I modified the code to skip the calculation as you suggested.

BenjaminBossan

Thanks for the new updates. We just merged another LoRA variant, which created merge conflicts with your PR, but it should be easy to resolve. Could you please take care? Thanks.

BenjaminBossan · 2025-12-08T11:13:40Z

tests/test_initialization.py

+        config1 = LoraConfig(
+            r=8,
+            target_modules=["linear"],
+            init_lora_weights=True,


You can remove this line, as it's irrelevant.

BenjaminBossan · 2025-12-08T11:13:45Z

tests/test_initialization.py

+        config2 = LoraConfig(
+            r=16,
+            target_modules=["linear"],
+            init_lora_weights=True,


You can remove this line, as it's irrelevant.

# src/peft/tuners/lora/model.py if len(self.peft_config) > 1: kasa_count = sum(1 for cfg in self.peft_config.values() if cfg.use_kasa) non_kasa_count = len(self.peft_config) - kasa_count if kasa_count > 0 and non_kasa_count > 0: raise ValueError("KaSA adapters cannot be mixed with other adapter types.")

I understood this to mean that since it's handled in this section, it's irrelevant elsewhere. Is my understanding correct?

Oh, this was a misunderstanding. I meant that the single line I commented on (init_lora_weights=True,) can be removed, the test as a whole is good to keep :) Please restore these tests.

ah okay haha

I changed the tests back :) !

…tLoraInitialization, simplifying the test suite and focusing on essential compatibility checks.

iambogeumkim · 2025-12-08T13:54:41Z

I applied what you mentioned and resolvd conflicts. Please take a look!

…dapter types in TestLoraInitialization, ensuring compatibility checks are enforced in both configurations.

iambogeumkim added 2 commits May 18, 2025 01:37

Add KaSA implementation to layer.py

387ad2a

Add use_kasa argument to LoraConfig

ae00a34

iambogeumkim marked this pull request as draft August 2, 2025 05:45

BenjaminBossan requested changes Aug 5, 2025

View reviewed changes

iambogeumkim added 3 commits September 2, 2025 20:54

Add use_kasa parameter to Linear class

6588f1a

Add KasaLinearVariant class (just copy of DoraLinearVariant class) in…

a824ac9

… variants.py

Add kasa description

05e4e07

iambogeumkim added 5 commits September 5, 2025 02:37

Remove unnecessary self.kasa

d1e7e43

[WIP] update KasaLinearVariant class with SVD implementation

9e53b9f

Modify merge/unmerge method in KasaLinearVariant class

aa37111

update KasaLinearVariant class with SVD implementation

9cfe65c

fix type in init method

f9d7cc7

BenjaminBossan requested changes Sep 9, 2025

View reviewed changes

iambogeumkim added 2 commits September 16, 2025 22:19

delete unnecessary part in layer.py

84813a3

add original reference in layer.py

39abcad

iambogeumkim closed this Sep 16, 2025

iambogeumkim force-pushed the peft-kasa branch from 39abcad to 20a9829 Compare September 16, 2025 16:53

merge main to peft-kasa

06f76d8

BenjaminBossan reopened this Sep 17, 2025

re-add KaSA implementation to variants.py

0043ae3

BenjaminBossan requested changes Oct 9, 2025

View reviewed changes

iambogeumkim and others added 7 commits October 11, 2025 15:37

Update tests/test_custom_models.py

0377170

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

Update src/peft/tuners/lora/variants.py

a536bbf

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

Update src/peft/tuners/lora/variants.py

f8d8057

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

Update src/peft/tuners/lora/variants.py

cd57c7b

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

add use_kasa param in LoraModel class

2431a2c

restore output_tensor variable in Linear class get_delta_weight method

7276b3b

add use_kasa handling condition in resolve_lora_variant method

5a67b1f

iambogeumkim marked this pull request as ready for review October 11, 2025 10:38

iambogeumkim added 2 commits October 12, 2025 00:50

fix KaSA self, mat1, mat2 dtype error

3ec6b18

fix make style error

461a89c

BenjaminBossan reviewed Oct 13, 2025

View reviewed changes

iambogeumkim added 3 commits October 18, 2025 15:23

add _skip_test_disable_adapters function

6ed64c1

add KaSA compatibility test

92061dd

update _check_new_adapter_config method with KaSA

9a2cf71

BenjaminBossan requested changes Nov 5, 2025

View reviewed changes

src/peft/tuners/lora/model.py Show resolved Hide resolved

tests/test_initialization.py Outdated Show resolved Hide resolved

tests/test_initialization.py Outdated Show resolved Hide resolved

iambogeumkim and others added 4 commits December 6, 2025 14:34

Implement tests to ensure KaSA adapters cannot be mixed with other ad…

39cf1f9

…apter types, enhancing compatibility checks in the initialization process.

Refactor KaSA adapter compatibility check to simplify logic and impro…

283ff0a

…ve readability in LoraModel class.

Refactor KasaLinearVariant class to improve code readability and ensu…

bfe8996

…re SVD is applied only once, while also cleaning up whitespace in multiple locations.

Merge branch 'main' into peft-kasa

cbc5b0c

BenjaminBossan reviewed Dec 8, 2025

View reviewed changes

iambogeumkim and others added 2 commits December 8, 2025 22:36

Merge branch 'main' into peft-kasa

14fa9d7

Remove tests for mixing KaSA adapters with other adapter types in Tes…

b6aae1e

…tLoraInitialization, simplifying the test suite and focusing on essential compatibility checks.

Add tests to validate that KaSA adapters cannot be mixed with other a…

951b6b2

…dapter types in TestLoraInitialization, ensuring compatibility checks are enforced in both configurations.

	def _skip_tests_with_multiple_adapters_with_target_parameters(config_cls, config_kwargs):
	if (config_cls == LoraConfig) and config_kwargs.get("target_parameters"):
	pytest.skip("LoRA with multiple adapters with target_parameters is not supported")

[WIP] Update LoraConfig for KaSA implementation #2698

Are you sure you want to change the base?

[WIP] Update LoraConfig for KaSA implementation #2698

Uh oh!

Conversation

iambogeumkim commented Aug 2, 2025

Uh oh!

BenjaminBossan left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Sep 1, 2025

Uh oh!

BenjaminBossan commented Sep 1, 2025

Uh oh!

iambogeumkim commented Sep 2, 2025

Uh oh!

BenjaminBossan commented Sep 2, 2025

Uh oh!

iambogeumkim commented Sep 4, 2025

Uh oh!

iambogeumkim commented Sep 8, 2025

1. variants.py

2. layer.py

Uh oh!

BenjaminBossan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iambogeumkim commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BenjaminBossan commented Sep 17, 2025

Uh oh!

iambogeumkim commented Sep 17, 2025

Uh oh!

BenjaminBossan commented Sep 18, 2025

Uh oh!

BenjaminBossan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

iambogeumkim commented Oct 11, 2025

1. 10 failures occurred

2. dtype issues (all resolved ✅)

3. AssertionError issues

Uh oh!

BenjaminBossan left a comment

Choose a reason for hiding this comment

Uh oh!

iambogeumkim commented Nov 4, 2025

1. _skip_test_disable_adapters function

2. TestKasaInitialization class

3. Logic to avoid repetitive SVD processes (Just an idea 💡)

Uh oh!

BenjaminBossan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

[WIP] Update `LoraConfig` for KaSA implementation #2698

[WIP] Update `LoraConfig` for KaSA implementation #2698

iambogeumkim commented Sep 16, 2025 •

edited

Loading

1. `_skip_test_disable_adapters` function

2. `TestKasaInitialization` class

iambogeumkim commented Nov 29, 2025 •

edited

Loading