Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new command: minikube cp to copy files into minikube #10198

Merged
merged 2 commits into from
Mar 23, 2021

Conversation

anencore94
Copy link
Contributor

@anencore94 anencore94 commented Jan 21, 2021

Add New Feature which represents scp command

Until now, it is possible to copy specific local file to specific path in minikube basically.
I mean 'basically' since it is possible to only file not directory.
And there are many more remaining tasks until now. (remaining problems are written in source code as //TODO)

Fix #9930

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 21, 2021
@k8s-ci-robot
Copy link
Contributor

Hi @anencore94. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 21, 2021
@anencore94
Copy link
Contributor Author

anencore94 commented Jan 21, 2021

Before coding some other feature such as 'enable -r option to copy directory', 'enable progress bar option' and the others, I want to get some reviews from you guys to check whether I am going the right way.

I also wonder how to implement the test code for this. Is it ok make only functional test for scp cmd at minikube/test/integration/functional_test.go ? or somewhere else? Please let me know.

Thanks a lot in advance!

@anencore94
Copy link
Contributor Author

/assign @tstromberg

@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@anencore94 anencore94 changed the title [WIP][Feature] Add minikube scp command [WIP] Add minikube scp command as new feature Jan 22, 2021
Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove TODO comments and add integration test then it will be good to do !

@medyagh
Copy link
Member

medyagh commented Feb 15, 2021

@anencore94 sorry for the late reply on this PR, this sounds like a good PR, lets remove the TODO notes and yes lets add an integration test to our Funcitonal Test

it could be a subtest simmilar to other tests


	// Parallelized tests
	t.Run("parallel", func(t *testing.T) {
		tests := []struct {
			name      string
			validator validateFunc
		}{
			{"ConfigCmd", validateConfigCmd},
			{"DashboardCmd", validateDashboardCmd},
			{"DryRun", validateDryRun},
// ...add your test here
			{"ScpCmd", validateScp},
..

and u can make a dummy hello world file and try to copy it using scp and validate the file was copied by

minikube ssh cat /tmp/hello_scp.txt

@anencore94 anencore94 force-pushed the minikube-scp branch 2 times, most recently from ae54fe2 to 5719576 Compare February 16, 2021 10:21
@anencore94 anencore94 changed the title [WIP] Add minikube scp command as new feature Add minikube scp command as new feature Feb 16, 2021
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 16, 2021
@anencore94 anencore94 force-pushed the minikube-scp branch 5 times, most recently from 245fda7 to 12fa99c Compare February 17, 2021 04:30
@anencore94
Copy link
Contributor Author

anencore94 commented Feb 17, 2021

Thanks for your review @medyagh !

I added the functional test for scp cmd as your comment. Could you re-check my PR?

  • the functional test for "PR / functional_docker_windows (pull_request)" and "PR / functional_hyperv_windows (pull_request)" failed since the scp command does not support on windows i guess. (or because filepath format is different)
    So I made the functional test to skip when the os is windows.

@anencore94 anencore94 requested a review from medyagh February 17, 2021 07:25
Copy link

@priyawadhwa priyawadhwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of shelling out to the scp command line tool directly, let's instead copy files into minikube the way we do in other parts of the code (for things like preload, or kubernets binaries). Instead of scp:

  1. Create a FileAsset with the NewFileAsset function
  2. Using a runner, run the built in Copy() function to copy the file asset into minikube

This way anyone can copy files in, regardless of if they have scp installed. This should also allow this command to run across all platforms, not just Linux. So, you shouldn't have to skip the functional test anymore.

Example of this in the code:

fa, err := assets.NewFileAsset(tarballPath, targetDir, targetName, "0644")
if err != nil {
return errors.Wrap(err, "getting file asset")
}
t := time.Now()
if err := r.Runner.Copy(fa); err != nil {
return errors.Wrap(err, "copying file")
}

If you need any more examples, searching for any call to asset.NewFileAsset should get you there.

@medyagh
Copy link
Member

medyagh commented Feb 24, 2021

Instead of shelling out to the scp flag directly, let's instead copy files into minikube the way we do in other parts of the code (for things like preload, or kubernets binaries). Instead of scp:

  1. Create a FileAsset with the NewFileAsset function
  2. Using a runner, run the built in Copy() function to copy the file asset into minikube

This way anyone can copy files in, regardless of if they have scp installed. This should also allow this command to run across all platforms, not just Linux. So, you shouldn't have to skip the functional test anymore.

Example of this in the code:

fa, err := assets.NewFileAsset(tarballPath, targetDir, targetName, "0644")
if err != nil {
return errors.Wrap(err, "getting file asset")
}
t := time.Now()
if err := r.Runner.Copy(fa); err != nil {
return errors.Wrap(err, "copying file")
}

If you need any more examples, searching for any call to asset.NewFileAsset should get you there.

@anencore94 I agree with @priyawadhwa lets do that

@priyawadhwa
Copy link

Since we're no longer using scp I would probably also rename this command to minikube cp (matches docker so is more intuitive)

@anencore94 anencore94 force-pushed the minikube-scp branch 2 times, most recently from 4da3879 to 2e1635b Compare February 25, 2021 07:29
@anencore94 anencore94 changed the title Add minikube scp command as new feature Add minikube cp command as new feature Feb 25, 2021
@anencore94
Copy link
Contributor Author

@priyawadhwa I changed the implementation of scp by copy with your review comments, Thanks!
By the way, it seems the function Copy of Runner doesn't support directory copy on its own. So If we want to copy directory, I guess we have to make directory and copy every files in for loop or some other way with Copy function.
Thus, for now, I just implement "minikube cp" for only file.

I want to implement the directory copying feature with another PR. (or someone else could do this)

Copy link

@priyawadhwa priyawadhwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple more comments, this is almost there!


// cpCmd represents the cp command, similar to docker cp
var cpCmd = &cobra.Command{
Use: "cp <source file path> <target file path followed by '/home/docker/'>",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused by this description, I thought the command would work something like:

minikube cp a.txt. /b.txt

Where does the /home/docker come in?

Additionally, we should probably insist that the second argument is an absolute path.

Copy link
Contributor Author

@anencore94 anencore94 Feb 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you the prefix_path '/home/docker/' is confusing :)
By the way, when we do 'minikube ssh', the starting path is '/home/docker' since the default username is docker.
Is it ok to copy on '/somewhere' where the permission is only for 'root user' ?

I think there maybe 3 options: (or you have any idea, please let me know :) )

  1. minikube cp a.txt /home/docker/b.txt
  • make minikube user to write '/home/docker' to see exactly where the path is.
  • But, they should write the /home/docker path for every copying
  1. minikube cp a.txt /b.txt
  • enable copy at root permissioned directory
  1. minikube cp a.txt b.txt
  • minikube user doesn't care about the user name, but always will be at /home/docker/b.txt as what i've implemented with this version

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What Option would be the best ? @priyawadhwa
In my opinion for now, option 1 or option 3 will be safe.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should ask the user for an absolute path and try to copy to wherever they want the file copied. If it fails because the command requires sudo, then we should provide a --force or -f flag which will copy over the file with root permissions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your comment.
I've changed to minikube copy can copy anywhere in minikube as your opinion.
Could you re-check? @priyawadhwa

cmd/minikube/cmd/cp.go Show resolved Hide resolved
Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anencore94 can u plz rebase the PR (pull upstream) ?

@medyagh
Copy link
Member

medyagh commented Mar 6, 2021

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 6, 2021
@minikube-pr-bot
Copy link

kvm2 Driver
error collecting results for kvm2 driver: timing run 0 with Minikube (PR 10198): timing cmd: [/home/performance-monitor/.minikube/minikube-binaries/10198/minikube start --driver=kvm2]: starting cmd: fork/exec /home/performance-monitor/.minikube/minikube-binaries/10198/minikube: exec format error
docker Driver
error collecting results for docker driver: timing run 0 with Minikube (PR 10198): timing cmd: [/home/performance-monitor/.minikube/minikube-binaries/10198/minikube start --driver=docker]: starting cmd: fork/exec /home/performance-monitor/.minikube/minikube-binaries/10198/minikube: exec format error

anencore94 added 2 commits March 8, 2021 10:51
- add new feature represents cp local file into minikube
- add functional test for cp command

Signed-off-by: anencore94 <[email protected]>
- only absolute path is allowed

Signed-off-by: anencore94 <[email protected]>
@anencore94
Copy link
Contributor Author

@anencore94 can u plz rebase the PR (pull upstream) ?

Thanks, I've done @medyagh

@medyagh
Copy link
Member

medyagh commented Mar 23, 2021

thank you @anencore94

@medyagh medyagh merged commit 3e869e0 into kubernetes:master Mar 23, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: anencore94, medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 23, 2021
@medyagh medyagh changed the title Add minikube cp command as new feature new command: minikube cp to copy files into minikube Mar 23, 2021
@anencore94 anencore94 deleted the minikube-scp branch March 24, 2021 04:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

minikube scp
7 participants